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EDITORIAL 


International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE) is an 
open access international peer-reviewed, open-access journal, which provides a platform for highlighting 
and discussing various cognitive science issues dealing with the problems of cognition (and its evolution) 
within some specific subject field - philosophical, psychological, linguistic, mathematical, psychogenetic, 
pedagogical, ergonomic. Editorial Board strives to provide a possibility for the scientists of different fields 
to publish the results of their research, technical and theoretical studies. ISCRSEE is multidisciplinary 
in approach, and will publish a great range of papers: reports of qualitative case studies, quantitative 
experiments and surveys, mixed method studies, action researches, meta-analyses, discussions of 
conceptual and methodological issues, etc. INCRSEE publisher is The Association for the Development 
of Science, Engineering and Education, Vranje, Serbia. Co-publishers are: University Business Academy, 
Faculty of Economics and Engineering Management in Novi Sad, Serbia and Don State Technical 
University, Rostov on Don, Russian Federation. 

JCRSEE particularly welcomes articles on the results of scientific research in various fields of 
cognitive science (psychology, artificial intelligence, linguistics, philosophy and neuroscience) catering 
for international and multidisciplinary audience. Readers include those in cognitive psychology, special 
education, education, adult education, educational psychology, school psychology, speech and language, 
and public policy.|IJCRSEE has regular sections: Original Research, Review Articles, Studies and articles, 
Book Reviews, Case Studies, and is published three times a year. This journal provides an immediate 
open access to its contents, which makes research results available to the public based on the global 
exchange of knowledge. The journal also offers access to uncorrected and corrected proofs of articles 
before they are published. 

The main aim of the Journal is to discuss global prospects and innovations concerning major issues 
of cognitive science, to publish new scientific results of cognitive science research, including the studies 
of cognitive processes, emotions, perception, memory, thinking, problem solving, planning, education 
and teaching, language and consciousness study, the results of studying man’s cognitive development 
and the formation of basic cognitive skills in everyday life. The Journal seeks to stimulate the initiation of 
new research and ideas in cognitive science for the purpose of integration and interaction of international 
specialists in the development of cognitive science as interdisciplinary knowledge. 

All articles are published in English and undergo a peer-review process. 

The scope of IJCRSEE is focused on cognitive research both in topics covered as well as disciplinary 
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Cognitive Psychology 
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Text Processing and Cognitive Technologies 


Curriculum Development 
Development of Learning Environment 
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Educational Technology 

Elementary Education 

Innovative Pedagogical Models 
Learning Systems Platforms 

Media Education 

Science Education 

Teaching and Learning Technologies 
Artificial intelligence and education 


JCRSEE has an international editorial board of eminent experts in their field from Russia, USA, 
Republic of Macedonia, Germany, Hong Kong, Greece, Serbia, Australia, United Kingdom, USA, Turkey, 
Nigeria, Bulgaria, Romania, Spain, Italy, Republic of Srpska, Croatia, Kingdom of Saudi Arabia (KSA), 
India, China, Thailand, Israel, Malaysia, Morocco, Jordan,, Iran... We are confident that INCRSEE will 
attract a great number of editors, eminent scientists in the field. The selection will be based on the activities 
of the editors and their desire to contribute to the development of the journal. 

IJCRSEE provides a platform for academics and scientists professionals to refer and discuss 
recent progress in the fields of their interests. Authors are encouraged to contribute articles which are not 
published or not under review in any other journal. 

Each submitted manuscript is evaluated on the following basis: the originality of its contribution 
to the field of scholarly publishing, the soundness of its theory and methodology, the coherence of its 
analysis, its availability to readers (grammar and style). Normal turn-around time for the evaluation of 
manuscripts is one to two months from the date of receipt. 

Submission of an original manuscript to the journal will be taken to mean that it represents original 
work not previously published, that is not being considered elsewhere for publication; that the author is 
willing to assign the copyright to the journal as per a contract that will be sent to the author just prior to the 
publication and, if accepted, it will be published in print and online and it will not be published elsewhere 
in the same form, for commercial purposes, in any language, without the consent of the publisher. 

The names and email addresses entered in this journal site will be used exclusively for the stated 
purposes of this journal and will not be made available for any other purpose or to any other party. 

The requirement for the submission of a paper implies that it has not been published before; that 
it is not under consideration for publication anywhere else; that its publication has been approved by all 
co-authors. 

When considering submitting an article, the Editors have provided the following criteria to assist 
authors with preparing their submissions: 

Originality - The author should ensure that the manuscript has not been previously published nor 
is being considered by another journal. 

Plagiarism - Content should be properly referenced. Be sure to check the paper for possible 
accidental plagiarism. Some plagiarism checker websites include: _http:/Avww.ithenticate.com/, www. 
antiplagiat.ru, www.grammarly.com, www.plagtracker.com_or www.duplichecker.com 

Writing — Please write in good English (American or British usage is accepted, but not a mixture of 
these). For non-native English speakers, and perhaps even for some native English speakers, grammar, 
spelling, usage, and punctuation of the texts are very important for an effective presentation. Hence, 
manuscripts are expected to be written in a clear, cogent, and readily understandable by an international 
readership. 

Manuscripts must be submitted online. Electronic submission reduces the editorial processing and 
reviewing time. As part of the submission process, authors are required to check off their submission 
compliance with all of the following items, and submissions may be returned to authors who do not adhere 
to the following guidelines: 

The submission has not been previously published or presented to another journal for consideration 
(or an explanation has been provided in Comments to the Editor). 

The submission file is in OpenOffice, Microsoft Word, RTF, or WordPerfect document file format. 

Where available, DOls and URLs for the references have been provided. 

The text is single-spaced; uses a 12-point font; employs italics, rather than underlining (except 
with URL addresses); and all illustrations, figures, and tables are placed within the text at the appropriate 


points, rather than at the end. 

The text adheres to the stylistic and bibliographic requirements outlined in the Author Guidelines. 

If submitting to a peer-reviewed section of the journal, the instructions in Ensuring a Double Blind 
Review have been followed. 

Amanuscript goes through the peer review process. Authors submit manuscripts to Editorial office 
via the online system. The acknowledgement letter should be sent to the author to confirm the receipt 
of the manuscript. The Chief Editor first reviews manuscripts. Chief Editor is assisted by Section Editors 
(could also be Co- or Associated Editors). The Editor assigns a Section Editor to see the manuscript 
through the complete review process and return it with a recommendation or decision. The manuscript 
is checked to see if it meets the scope of the Journal and its formal requirements. If it is incorrect or 
unsuitable, the author should be informed and the manuscript filed (or returned if requested) — direct 
rejection. Manuscripts that are not suitable for publication in the Journal are rejected. A Rejection letter 
is sent to the author stating the reason for rejection. If the manuscript conforms to the aims and scope of 
the Journal, and formally abides by the Instructions to Authors it is sent out for review. Depending on the 
type of paper, it could be accepted immediately for publication (invited Editorial, Book review etc) by the 
Chief Editor. 

Check that the manuscript has been written and styled in accordance with the Journal style; that 
it carries an abstract (if applicable), keywords, correct reference system etc. and check that the correct 
blinding system has been used. If anything is missing ask the author to complete it before the manuscript 
is sent out for review. 

The manuscript is sent out for review. The reviewer reads and evaluates the manuscript and 
eventually sends a review report to the Chief Editor. The time for review can be set to 2-6 weeks depending 
on the discipline (more time is usually given to papers in the humanities and social sciences). Make sure 
to provide the reviewer with clear instructions for the work, e.g. outlined in the form of a Review report or 
a number of questions to be considered. 

Based on the reviewers’ comments the Chief Editor makes a decision to: 

* Accept the manuscript without further revision 

* Accept after revision 

* Ask authors to resubmit 

* Reject 

An acceptance letter is sent to the author and the final manuscript is forwarded to production. 
Sometimes, the authors are requested to revise in accordance with reviewers’ comments and submit 
the updated version or their manuscript to the Chief Editor. The time for review can be set to 2-6 weeks 
depending on the discipline and type of additional data, information or argument required. The authors 
are requested to make substantial revisions to their manuscripts and resubmit for a new evaluation. A 
rejection letter is sent to the author and the manuscript is archived. Reviewers might be informed about 
the decision. 

After review a manuscript goes to the Copy Editor who will correct the manuscript concerning the 
correct referencing system, confirmation with the journal style and layout. When Copy Editor finishes his/ 
her work they send manuscripts to the Layout editor. 

Layout Editor is responsible for structuring the original manuscript, including figures and tables, into 
an article, activating necessary links and preparing the manuscript in the various formats, in our case PDF 
and HTML format. When Layout Editor finishes his/her job they send manuscripts to Proof Editor. 

Proof Editor confirms that the manuscript has gone through all the stages and can be published. 

This issue has 13 articles (11 Original researches and 2 Review articles). Our future plan is to 
increase the number of quality research papers from all fields of science, engineering and education. The 
editors seek to publish articles from a wide variety of academic disciplines and substantive fields; they are 
looking forward to substantial improvement of educational processes and outcomes. 
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Abstract: This study aimed to assess the influence of students’ intrapersonal factors, namely Academic Intrinsic Motivation 
(AIM), Perceived Creativity Fostering Teacher Behavior (P-CFTB), Academic Self-Efficacy (ASE), and Self-Regulated Learning 
(SRL) on student engagement in undergraduate psychology students taking online Statistics courses. A cross-sectional and 
quantitative design was used from October to December 2022. The data collection procedure used a convenience sampling 
technique, with questionnaires distributed online (via social media) and offline (via lecturers, the Student Executive Board, and 
the Association of the Faculty of Psychology from various universities in Indonesia). The research participants were psychology 
undergraduates who had studied and passed the Statistics courses online, with 671 filling out the questionnaire. The results 
showed that all students’ intrapersonal factors, namely AIM, P-CFTB, ASE, and SRL, can determine student engagement by 
66.9%, with ASE having the highest influence (23.99%) and P-CFTB having the lowest impact (9.78%). Moreover, the correlation 
value between SRL and SE was r = 0.700, p < 0.001, signifying a robust positive relationship between both variables. 

Keywords: student engagement, perception of creativity fostering teacher behavior, academic self-efficacy, academic 
intrinsic motivation, self-regulated learning, online learning. 


Introduction 


Low student engagement reduces students’ chances of acquiring the necessary talents and skills 
(Kuh, 2009). Besides students’ ability to apply knowledge in more complex situations (Primana, 2015), 
students’ memory of learning materials and final grades will also be low (Staikopoulos et al., 2015). 
Thus, student engagement has become a critical issue in higher education as it significantly influences 
the quality of learning students acquire (Staikopoulos et al., 2015; Xia et al., 2022). Moreover, due to 
COVID-19, Higher Education Institutions (HEIs) worldwide were forced to apply online learning methods. 
One of the issues often highlighted in online learning methods is closely related to student engagement 
(Czerkawski and Lyman, 2016; Xia et al., 2022). According to previous studies, online learning methods 
could increase student engagement (Khusniyah and Hakim, 2019; Kuntarto, 2017). On the contrary, 
studies in Indonesia found that online learning methods reduced student engagement (Fatoni et al., 2020; 
Rusman and Nasution, 2020; Sa’diyah, 2021). 

Student engagement only happens when students involve their feelings and active thinking 
processes in learning (Harper and Quaye, 2009). Fatoni et al. (2020) found that 100 students from five 
universities in Indonesia experienced student engagement problems during online learning. This finding is 
supported by Rusman and Nasution (2020) on UIN Sumatera Utara Medan college students, who found 
that out of 191 students, only 4.71% had high student engagement during online learning. Furthermore, 
Sa’diyah (2021) found that students only join online classes to fulfill their attendance but ignore the lessons 
and do other activities. Thus making them have low student engagement. 

Maulana and Iswari (2020), who analyzed student engagement in calculation-based courses, found 
that online learning in courses (such as Statistics) causes students to experience stress and difficulty 
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understanding the learning material. It made them have low student engagement scores. Moreover, 
students often view Statistics courses as very mathematical, challenging, and frightening because they 
use many formulas (Carpenter and McDonald, 2017; Waruwu, Hao and Hia, 2022; Zaimil, 2017). 

Stress regarding statistics courses also occurs in psychology students. Suminta (2016) found 
psychology students are prone to experience anxiety when studying Statistics. For psychology students, 
besides being difficult, the Statistics courses are also felt to be unrelated to their future career choices 
(Lloyd and Robertson, 2012). In fact, those courses are one of the fundamental courses in the psychology 
program. From developing new therapy techniques to evaluating the effectiveness of strategies, it is a 
statistical analysis that plays a role in providing an overview and drawing conclusions. Psychologists use 
statistical analysis to find ways to interpret and draw conclusions from their data (Watts and Thomas, 
2022). Given the importance of Statistics for Psychology students, it is urgent to explore the factors that 
can affect student learning engagement (Firmansyah, 2017; Ulpah, 2009). 

Several studies exploring factors affecting student engagement in higher education have been 
conducted. Almarghani and Mijatovic’s (2017) study showed that the lecturers’ role and teaching skills 
are the most influential factors for student engagement. Elshami et al. (2022) explored that factors such 
as techno-pedagogical skills, self-directed learning, peer-assisted learning, and collaborative learning are 
required to support medical and health students’ engagement in online learning. Calabrese et al. (2022) 
found that frequency and regular meetings, demographic factors such as course of study, expectation, 
and perception of students, can affect student engagement in personal tutoring schemes. 

Student engagement is an interaction process between contextual or learning environment and 
intrapersonal factors (Christenson, Reschly and Wylie, 2012). Contextual factors include the context of 
teaching and social relationships (support from teachers, friends, and parents) (Christenson, Reschly and 
Wylie, 2012; Skinner, Kindermann and Furrer, 2009). However, several studies have found that contextual 
factors are not always able to predict student engagement. In fact, prior research had shown that students’ 
perceptions of understanding contextual factors were predictive for student engagement (Christenson, 
Reschly and Wylie, 2012; van Petegem et al., 2007). The students’ perceptions of these contextual factors 
belong to intrapersonal factors, such as how they perceive their learning experience and the source of 
knowledge (Christenson, Reschly and Wylie, 2012; Raviv et al., 2003). 

Various intrapersonal factors have been reported to influence student engagement, such as 
academic self-efficacy (ASE) (Helsa and Lidiawati, 2021; Pramisjayanti and Khoirunnisa, 2022; Zhen et 
al., 2017), academic intrinsic motivation (AIM) (Dierendonck et al., 2023; Myint and Khaing, 2020), and 
self-regulated learning (SRL) (Lidiawati and Helsa, 2021; Setiani and Wijaya, 2020). ASE is a student’s 
determination and behavior toward assignments and the educational process (Chang and Chien, 2015; 
Zhen et al., 2017). Students with low ASE showed more indifference and low engagement in learning 
(Bassi et al., 2007). In contrast, according to meta-analysis studies (Chang and Chien, 2015), students 
with high ASE scores had higher student engagement. 

AIMis also strongly related to engagement. This is proven by Dierendonck et al. (2023), who found 
that students who study for intrinsic reasons tend to be more focused and actively engaged because 
they enjoy learning. The next intrapersonal factor that is positively related to student engagement is self- 
regulated learning (SRL) (Lidiawati and Helsa, 2021; Setiani and Wijaya, 2020). To improve academic 
achievement, students must have self-regulated learning skills to stay engaged in lectures, especially 
online learning. 

Additionally, Primana (2015) found that college students in Indonesia perceive their lecturers as 
the primary source of knowledge. Students’ perceptions of their lecturers also significantly impact student 
engagement (Pachler, Kuonath and Frey, 2019; Primana, 2015). Furthermore, Lawton and Taylor (2020) 
investigated college student perceptions of engagement and teaching strategies in the Introduction to 
Statistics course. They discovered activities that increased student engagement in online learning, such 
as when lecturers gave simulation-based instructions and asked students to carry out group discussions 
and independent study. On the other hand, students identified low engagement when there were no 
hands-on activities and students were only taking notes and listening during online classes. 

A strategy and consistent effort of lecturers to encourage students to use their knowledge to think 
independently and flexibly by using new approaches to solve problems is called Creativity Fostering 
Teacher Behavior (CFTB) (Cropley, 1997). Based on Lawton and Taylor (2020) findings, we concluded 
that the concept of creativity fostering teacher behavior (CFTB) was strongly related to how students 
perceived the teaching strategies used by lecturers in the study. CFTB is a teaching strategy that aims to 
develop students’ creative thinking or behavior” (Jeffrey and Craft, 2004). The CFTB concept also aligns 
with learning material in Statistics courses, which require flexible and creative thinking to solve complex 
calculation problems (Grégoire, 2016). 
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Up until now, most studies involving the CFTB variable measured CFTB from teachers’ or lecturers’ 
perceptions (Huang, 2022; Karwowski, Gralewski and Szumski, 2015; Palaniappan, 2009; Varatharaj, 
2018); pre-service teachers (Katz-Buonincontro, Perignat and Hass, 2020; Kim et al., 2019; Orr and 
Kukner, 2015); and the effect of CFTB on students’ creativity (Bell et al., 2014; Hafizi and Kamarudin, 
2020; Mao et al., 2020; Zhang et al., 2022). Meanwhile, the research that examined the relationship 
and role of students’ perceptions of CFTB on student engagement is still limited. Even though, since 
2000, Soh has suggested that the research on CFTB should be measured based on student perceptions. 
Students’ perceptions of CFTB in this study will be called perceived CFTB or P-CFTB. 

As previous studies have shown, student engagement is more influenced by intrapersonal factors. 
In addition, as Rusman and Nasution (2020) stated, there is a need for research that explores the factors 
affecting student engagement in an online learning context. Therefore, it is vital to conduct exploratory 
research on students’ intrapersonal factors by assessing the role of these factors in learning engagement 
in online Statistics courses. Rodgers (2008) also said that to increase teaching effectiveness and academic 
achievement, HEls should consider developing online teaching strategies that encourage greater student 
engagement. 

Although some studies have investigated the role of AIM (Cayubit, 2022; Giesbers et al., 2013; 
Gettle, 2022), ASE (Chang and Chien, 2015; Helsa and Lidiawati, 2021; Pramisjayanti and Khoirunnisa, 
2022; Zhen et al., 2017), SRL (Lidiawati and Helsa, 2021; Nurfitri and Aslamawati, 2021; Setiani and 
Wijaya, 2020; Utami and Aslamawati, 2021) that influence student engagement, however their role in 
undergraduate psychology students taking online Statistics courses are still limited. Furthermore, empirical 
research on CFTB is limited to teacher/lecturer (Huang, 2022; Karwowski, Gralewski and Szumski, 2015; 
Palaniappan, 2009; Varatharaj, 2018) and pre-service teacher/lecturer (Katz-Buonincontro, Perignat and 
Hass, 2020; Kim et al., 2019; Orr and Kukner, 2015), so few studies examine student perceptions of 
CFTB. Therefore, the study hypothesizes that academic intrinsic motivation, perceived creativity fostering 
teacher behavior, academic self-efficacy, and self-regulated learning simultaneously can determine 
student engagement in undergraduate psychology students taking online Statistics courses. 


Materials and Methods 


Research Design 

This study used a cross-sectional design from October to December 2022. The following criteria 
were emphasized for the selection of participants: (1) The psychology undergraduates need to have 
studied and passed the Statistics course through the online learning method, (2) The students should 
follow the appropriate time interval for studying the course, and (3) The period used by the undergraduates 
to complete the research questionnaire should not be more than three semesters. It has been done 
since the implementation of online learning methods by most HEls in the last three semesters during 
the COVID-19 pandemic. The data collection procedure used a convenience sampling technique, with 
questionnaires distributed online (via social media) and offline (via lecturers, the Student Executive Board, 
and the Association of the Faculty of Psychology from various universities in Indonesia). 
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Table 1 
Demographic Data of Participants (N= 533) 
Demography n % 
Gender Female 435 81.61 
Male 98 18.39 
Age (year) 18 32 6.00 
19 232 «© 43.53 
20 199 37.34 
21 48 9.01 
>21 2 2 «413 
Semester 3 322 ~—-60.41 
4 3 0.56 
5 183 34.33 © 
ee ee ae 
8 1 0.19 
>8 3 0.56 
University Location (Province) | Aceh 4 0.19 
— Banten 36 6.75 
DI Yogyakarta 9 1.69 
DKI Jakarta 219 41.09 
West Java 163 30.58 
CentralJava is 9s 1:69 
Eastuava  ——«-«63._—s'11.82— 


South Kalimantan 5 0.94 
South Sulawesi 3 0.56 
South Sumatera 22 4.13 
North Sumatera 3 0.56 — 


Participants 

The research participants were psychology undergraduates who had studied and passed the 
Statistics courses via online learning, with 671 filling out the questionnaire. However, only 533 participants 
met the selection criteria and were spread from 11 provinces in Indonesia, with the majority originating 
from DKI Jakarta (40.68%). Most of the participants were 19 years old (43.55%), female (81.54%), and 
3rd-semester students (59.86%). A detail of participants’ demographic data of participants can be seen 
in Table 1. 


Research Instrument 

This research used five research instruments, namely (1) The University Students’ Engagement 
Inventory (USEI) by Morocco et al. (2016), (2) The Creativity Fostering Teacher Index (CFTIndex) by 
Soh (2000), (3) The Indonesian College Academic Self-Efficacy Scale (CASES) by Ifdil et al. (2019), (4) 
The Online Self-regulated Learning Questionnaire (OSLQ) by Mutiara and Rifameutia (2021), and (5) 
The Academic Motivation Scale (AMS) by Marvianto and Widhiarso (2019). Each instrument was tested 
for reliability through the Cronbach Alpha and CRiT values, with validity analyzed by using Confirmatory 
Factor Analysis (CFA). The fit index used as a criterion for the cut-off value was also CFI > 0.90, RMSEA 
< 0.08, and SRMR < 0.08 (Hu and Bentler, 1999; Schermelleh-Engel, Moosbrugger and Miller, 2003). 


Students’ Engagement (SE) 

The Indonesian version of the University Student Engagement Inventory (USEI) instrument by 
Morocco et al. (2016) was adopted to measure students’ engagement. This instrument consisted of three 
dimensions (cognitive, behavioral, and emotional engagement), each with five items. These items were 
then assessed using a 6-point Likert scale, with 1 to 6 emphasizing never to always, respectively. The 
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total value was counted by adding up the scores of each item. An example of a sample item is, “I usually 
do my homework on time.” The Indonesian version of the USEI had a reliability value of Cronbach’s a = 
0.862, signifying that the instrument was reliable (Kaplan and Saccuzzo, 2017). All items also showed 
good internal validity, with CRiT values ranging from 0.261-0.647 (Nunnally and Bernstein, 1994). In 
addition, CFA showed that the USEI was valid due to meeting the goodness of fit criteria with chi-square 
= 3.72 (X?= 320.355; df = 86, p < 0.001), CFI = 0.905, RMSEA = 0.072, and SRMR = 0.067. 


Academic Intrinsic Motivation (AIM) 

AIM was measured through the Academic Motivation Scale (AMS) by taking three dimensions 
of Intrinsic Motivation from Vallerand et al. (1992), which has been adapted into the Indonesian version 
by Marvianto and Widhiarso (2019). It consisted of three factors, namely IM-to know, IM-toward 
accomplishment, and IM-to experience stimulation, each having four items. Using a 6-point Likert scale, 
namely 1 (do not correspond at all) to 6 (corresponds exactly), the total scores of AIM were obtained 
by adding up the scores of all items. The sentences in several items were also adjusted, such as the 
replacement of ‘school’ with ‘college’ to fit the research context. For example, a sample item stated, 
“Because | experience pleasure and satisfaction while learning new things.” From the results, AMS had a 
reliability value of Cronbach's a = 0.896 and CRIT = 0.526 - 0.685. CFA also showed that AMS was valid 
due to meeting the goodness of fit criteria with chi-square = 4.17 (X?= 204.624: df= 49, p < 0.001), CFI = 
0.945, RMSEA = 0.077, and SRMR = 0.040. 


Perceived Creativity Fostering Teacher Behavior (P-CFTB) 

Researchers adapted the Creativity Fostering Teacher Index (CFTIndex) instrument by Soh (2000) 
into the Indonesian version to measure lecturer creativity fostering from a student perspective, named 
Perceived CFTIndex (P-CFTIndex). This instrument initially contained 45 items, which was reduced to 27 
after being adapted based on the CFTB-Index procedure by Lee and Kemple (2014). It still consisted of 
nine dimensions, each containing three items in the Indonesian version. Each item was also assessed 
using a 6-point Likert scale, namely 1 (never) to 6 (always). Moreover, the total score was obtained by 
adding up the scores of each item, with a sample example indicating the following, “Lecturer encourages 
me to try out what | have learned in different situations.” From the results, P-CFTIndex had a reliability 
value of Cronbach’s a = 0.944, signifying that the instrument was reliable. All items also showed good 
internal validity, with the CRiT values ranging from 0.357 to 0.713. In addition, CFA showed that P-CFTB 
measuring instrument was valid because of meeting the goodness of fit criteria with chi-square = 3.25 (X? 
= 936,660; df = 288, p < 0.001), CFI = 0.904, RMSEA = 0.065, and SRMR = 0.047. 


Academic Self-Efficacy (ASE) 

ASE was measured using the Indonesian CASES version by Ifdil et al. (2019), adapted from Owen 
and Froman (1988). This instrument initially contained 33 items, which were then reduced to 17 items 
and categorized into three dimensions. These dimensions included technical skills (5 items), overt social 
situation (6 items), and cognitive operation (6 items). The total score was calculated by adding up the 
scores of all items. Each item was also assessed by using a 6-point Likert scale, namely 1 (strongly 
disagree) to 6 (strongly agree), with a sample example presented as follows, “| master most of the 
lecture materials having many elements of calculation.” From the results, CASES had a reliability value 
of Cronbach’s a = 0.914, with CFA emphasizing its validity due to meeting the goodness of fit criteria with 
chi-square = 2.36 (X? = 1092.270:; df = 462, p < 0.001), CFI = 0.905, RMSEA = 0.079, and SRMR = 0.054. 


Self-Regulated Learning (SRL) 

Online Self-Regulated Learning Questionnaire (OSLQ) was used to measure SRL (Barnard-Brak, 
Paton and Lan, 2010) and adapted to the Indonesian version by Mutiara and Rifameutia (2021). This 
instrument initially contained 24 items, which were then reduced to 21 elements after translation and 
categorized into six dimensions. These dimensions included environmental structuring, goal-setting, time 
management, help-seeking, task strategies, and self-evaluation, which consisted of 4, 5, 3, 2, 4, and 
4 items, respectively. The items were also assessed using a 6-point Likert scale, namely 1 (strongly 
disagree) to 6 (strongly agree). Moreover, the total score was obtained by adding up the scores of each 
item, with the example of a sample stating the following, “I prepare questions before joining an online 
lecture-discussion session.” OSLQ had a reliability value of Cronbach’s a = 0.918 and CRIT = 0.377 - 
0.673. CFA also showed that the instrument was valid due to meeting the goodness of fit criteria with 
chi-square = 3.63 (X? = 626.008; df = 172, p < 0.001), CFI = 0.906, RMSEA = 0.070, and SRMR = 0.055. 
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Research Procedure and Data Analysis 

The procedures and instruments of this study were carefully reviewed by the Faculty of Psychology 
ethics committee under number 136/FPsi.Komite Etik/PDP.04.00/2022. The adaptation of measuring 
instruments into Indonesian versions (P-CFTIndex and USEI) was also carried out regarding the 
procedure of Beaton et al. (2000), which contained five stages, namely (1) translation, (2) synthesis, (3) 
back translation, (4) expert assessment, and (5) data collection. In addition, the results were analyzed by 
performing multiple regression through JASP software version 0.164. 


Results 


Statistic Descriptive Analysis 

Pearson's test was carried out to determine the correlation of each variable, as described in Table 
2. The results showed a moderate correlation among the variables, with ASE and SE showing the most 
vital relationship than other independent determinants (r = 0.714, p <.001). However, P-CFTB and SE 
exhibited a moderate correlation between the independent and the dependent variables (r = 0.593, p 
<.001), with P-CFTB and AIM portraying the weakest relationship (0.468). 

Table 3 explains the descriptive analysis of each variable, where the range of values included 
45-90, 37-72, 77-162, 90-193, and 45- 126 for SE, AIM, P-CFTB, ASE, and SRL, respectively. Based 
on the results, the Mean/Standard Deviation values were high for SE, AIM, and P-CFTB at 69.46/9.17, 
58.20/7.72, and 126.47/18.18, respectively. Meanwhile, the Mean/Standard Deviation values were in the 
moderate category for ASE and SRL at 71.90/12.63 and 88.92/16.18, respectively. 


Table 2. 
Variables Intercorrelation 
Variable 1 2 3 4 5 
—1.SE — 
(2AIM |e 
3.P-CFIB 9B ABB a Seer eee 
4. ASE 0.714*** | 0.505*** 0.495*** = | 
_5. SRL 0.700*** | 0.503*** 0.572*** 0.676*** = 


Note: * p< .05, **p<.01, *** p< .001 
SE = Students’ Engagement, AIM = Academic Intrinsic Motivation, P-CFTB = Perceived Creative Fostering Teacher 
Behavior, ASE = Academic Self-Efficacy, SRL = Self-Regulated Learning 


Table 3. 
Descriptive Statistic 

Variables | N Mean | SD Min Max 

1.5E 533 69.46 9.17 | 45.00 90.00 

2. AIM 533 58.20 7.72 37.00 72.00 

3.P-CFIB 933 126.47 0 88 00 162.00, 

3.ASE > — 6338 i 71.90 | — 1263 _ 9000 193.00 

4. SRL 533 88.92 16.18 45.00 126.00 


Note: SE = Students’ Engagement; AIM = Academic Intrinsic Motivation; P-CFTB = Perceived Creative Fostering 
Teacher Behavior; ASE = Academic Self-Efficacy; SRL = Self-Regulated Learning. 
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Table 4. 
Participants’ Categorization of Each Variable (N= 533) 
: Catego 
bce Low Moderate High 
| score range <40 | 40 <x<65 >65 
SE t= no ee O © BF 376 
% | 00.00 29.46 70.54 
_ scorerange S32 — 82sx<82 252, 
AIM n | 0 101 432 
% 00.00 18.95 81.05 
score range | ay iis 72<x<117 >117 
P-CFTB | no | Oo | — 1620C—C 371 
| % | 00.00 30.39 69.61 
_— scorerange 4G 453<x<73.7 2713.7 
ASE n | 11 283 239 
| % | 2.06 53.09 AA 84 
| score range <56 56 <x < 91 >91 
SRL — no ae 12 —28000—~Ct 2A 
% 02.25 52.53 45.22 


Note: SE = Students’ Engagement; AIM = Academic Intrinsic Motivation; P-CFTB = Perceived Creative Fostering 
Teacher Behavior; ASE = Academic Self-Efficacy; SRL = Self-Regulated Learning. 


Table 4 presents the participant categorization of each variable, where the majority of samples 
were observed in the high category for SE (70.54%), AIM (81.05%), ASE (50.47%), and P-CFTB at 
70.54%, 81.05%, 50.47%, and 69.61%, respectively. However, most participants on the SRL variable 
were included in the medium category at 52.53%. 


Multiple Linear Regression Prerequisite Test 

Multiple regression analysis was employed to determine whether the research model’s four 
independent variables (IVs) collectively possess predictive abilities for student engagement. When 
assessing multiple linear regression models, it is crucial to meet at least four prerequisite tests: 
multicollinearity, data linearity, homoscedasticity, and multivariate normality (Osborne and Waters, 2002). 


Multicollinearity Test 

The multicollinearity test was applied to gauge the degree of correlation among the independent 
variables. Midi, Sarkar and Rana (2010) stipulated that there is no multicollinearity when the tolerance 
value is >0.1 and the Variance Inflation Factor (VIF) is <10. Table 5 demonstrates the tolerance values for 
each variable are >0.1, and the VIF values for all variables are <10, signifying that the model meets the 
multicollinearity requirement. 


Table 5. 
Multicollinearity Test Result 
Collinearity Statistics 
Model Tolerance VIF 
Hi (Intercept) 
AIM 0.664 1.506 
P-CFTB 0.620 1.614 
ASE 0.499 2.005 
SRL 0.456 2.195 


a. Dependent variable: Student Engagement 
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Data Linearity Test 

The data linearity test establishes a linear relationship among the independent variables (Hayes, 
2015). It is tested by examining the scatterplot between the DV and each IV. The outcomes of the linearity 
test, illustrated in Figure 1 below, indicate the presence of a linear relationship between the dependent 
and independent variables, thereby fulfilling the linearity assumption. 


Homoscedasticity Test 

Ascatterplot of residuals against predicted values is used to evaluate homoscedasticity (Hariyanto, 
Triyono and Kohler, 2020). Figure 2, presented below, illustrates the data distribution pattern, signifying 
that the assumption of homoscedasticity has been met. 
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Figure 1. Data Linearity Test Results 


Residuals 


sjenpisay pazipsepueis 


Predicted Values 


Figure 2. Homoscedasticity Test Result 


Multivariate Normality Test 

The multivariate normality test is used to verify the normal distribution of data. Figure 3 indicates that 
the residuals conform to a normal distribution, affirming that the model meets the multivariate normality 
assumption. 
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Figure 3. Normal Distribution Curve 


Multiple Linear Regression Results 

Multiple linear regression was conducted after all the prerequisite tests showed that the results met 
the requirement. Table 6 below shows that F = 267.358: p < 0.001, meaning that AIM, P-CFTB, ASE, and 
SRL can significantly determine student engagement. This means the research hypothesis is accepted. 
Table 6 also shows that multiple linear regression’s coefficient of determination (R2) is 0.669. Based on 
the guidelines for interpreting the coefficient of determination (R2) by Sarjana, Hayati and Wahidaturrahmi 
(2020), 0.669 is included in the strong influence category. This means that AIM, P-CFTB, ASE, and SRL 
significantly predict learning engagement with an influence contribution of 66.90%, with the remaining 
33.10% influenced by variables not included in this study. 


Table 6. 
Multiple Linear Regression Model Summary 
Model R PR Adjusted R? = RMSE R? Change FChange  df1_ df2 p 
Ho 0.000 ~—:0.000 0.000 9.168 0.000 1 0) $32 
Hy 0.818 ~=—-:0.669 0.667 5.291 0.669 267.358 4 528 < .001 
Table 7. 
Coefficients 
Model ae a | Standard Error ara t p 
Ho (Intercept) 69.460 0.397 174.919 | <.001 
Hi (Intercept) 11.539 1.969 5.859 < .001 
AIM 0.292 0.036 0.246 8.022 < .001 
PCFTB 0.083 0.016 0.165 5.188 =< .001 
ASE 0.244 0.026 0.336 9.477 < 001 
SRL 6014500 i022 iS i BBS < 001 


a. DV: Student Engagement . 
Note: SE = Students’ Engagement; AIM = Academic Intrinsic Motivation; P-CFTB = Perceived Creative Fostering 
Teacher Behavior; ASE = Academic Self-Efficacy; SRL = Self-Regulated Learning. 


From Table 7 above, the relationship between variables can be seen in the following equation: 
Y =a + B1X1 + B2X2 + B3X3 + B4X4 +e 
Y (SE) = 11.539 + 0.292*AIM + 0.083*PCFTB + 0.244*ASE + 0.145*SRL + € 
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From the multiple linear regression equation above, it can be explained as follows: 

a) The constant (a) has a positive value of 11.539. This shows that if all independent variables are 
worth 0, the base value of student engagement is 11.539. 

b) For every percentage increase in AIM, student engagement increases by 0.292, assuming other 
independent variables remain constant. 

c) For every percentage increase in P-CFTB, student engagement increases by 0.083, assuming 
other independent variables remain constant. 

d) For every percentage increase in ASE, student engagement increases by 0.244, assuming other 
independent variables remain constant. 

e) For every percentage increase in SRL, the student engagement increases by 0.145, assuming 
other independent variables remain constant. 


To determine the amount of influence of each independent variable on the dependent variable 
partially, we used the Beta*Zero Order formula. Based on the formula, it is known that the most significant 
influence comes from the ASE, with an influence contribution of 23.99%. This is followed by the SRL, 
which contributes an influence of 17.85%, the AIM of 15.27%, and the P-CFTB of 9.78%. 


Discussions 


This study provides an overview of the interaction between intrapersonal factors, namely 
Academic Intrinsic Motivation (AIM), Academic Self-Efficacy (ASE), Self-Regulated Learning (SRL), 
and Perceived Creativity Fostering Teacher Behavior (P-CFTB), in predicting student engagement in 
undergraduate psychology students taking online statistics courses. Hypotheses were examined to test 
whether these intrapersonal factors simultaneously affect student engagement. The results found that 
the four independent variables studied (AIM, ASE, SRL, and P-CFTB) significantly determined student 
engagement, with a contribution of 66.9%. Based on the research results above, it can be concluded 
that the study’s hypothesis is confirmed. These results support findings in previous studies that state that 
intrapersonal factors are the main factor in predicting student engagement (Christenson, Reschly and 
Wylie, 2012; van Petegem et al., 2007). 

Furthermore, we examined the power contribution of each independent variable to student 
engagement. The results showed that AIM significantly predicted SE among psychology undergraduates 
in online Statistics courses (R2=15.27%; p<0.001). The results supported Gettle (2022), where AIM 
significantly affected SE in psychology undergraduate students. The findings of this study also support 
the findings reported by Giesbers et al. (2013) and Gettle (2022), who discovered that intrinsic motivation 
is closely linked to student engagement in online learning by utilizing technologies in applications, such 
as chat, webcams, and microphones. The academic achievement of students is also strongly related to 
student engagement through the usage of these tools. 

Furthermore, ASE had the most significant influence on SE at 23.99%, compared to other variables. 
The findings support Warwick’s statement (Warwick's, 2008) that student self-efficacy predicts student 
involvement. Students who have confidence that they are capable will be more persistent in facing 
difficulties. On the other hand, students with low self-efficacy will feel helpless and less persistent in 
completing complex tasks. In line with this, research by Helsa and Lidiawati (2021) found that self-efficacy 
had an effect of 36.9% on student learning engagement. Pramisjayanti and Khoirunnisa’s (2022) study 
also found that self-efficacy influenced 64.9% of student engagement in online learning. 

The results also revealed that SRL significantly affected SE at an impact level of 17.85%. This was 
not in line with Lidiawati and Helsa (2021), where a more significant effect of SRL on SE was observed 
at 55.9%. Nurfitri and Aslamawati (2021) found that self-regulated learning has a 57% impact on student 
engagement. A similar result was also found in Utami and Aslamawati (2021), who explained that the 
effect of self-regulated learning was 52.8% on student engagement. These differences emphasized 
specific research measuring students’ SRL abilities when learning Statistics courses. 

Moreover, this study found that student perceptions of creativity fostering teacher behavior (P-CFTB) 
can predict student engagement in online learning, with a 9.78% contribution. Students’ perceptions of 
teaching strategies that encourage creativity, such as independent learning, opportunities to develop and 
share ideas, divergent thinking, reflection, learning opportunities with a variety of materials and conditions, 
and support for overcoming failures, play a significant role in predicting student engagement in students 
who take online Statistics courses. In line with previous research has found that students’ attitudes and 
beliefs about lecturers influence student engagement in the classroom (Christenson, Reschly and Wylie, 
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2012; Pachler, Kuonath and Frey, 2019; Primana, 2015; Raviv et al., 2003). 

Solving statistical problems requires the ability to think creatively. Therefore, creative teaching 
strategies (teaching for creativity) will also influence students’ attitudes toward the statistics learning 
process, and attitudes toward learning significantly positively affect learning achievement (Hu, Deng and 
Guan, 2016). This finding also expands on research by Golder (2018) and Lawton and Taylor (2020) 
regarding student perceptions of lecturer behavior. Golder (2018) found that students’ perceptions of their 
lecturers significantly related to their attitudes toward learning. Furthermore, Lawton and Taylor (2020) 
found that students’ perceptions of independent teaching and learning strategies can increase student 
engagement. 

Additionally, this research found that ASE strongly and significantly correlated with SE in statistics 
courses (r = 0.714, p < 0.001). This was not in line with this previous research; Fan and Williams (2010) 
showed that students’ ASE in mathematics and English significantly correlated with engagement in these 
subjects. The results confirmed that the correlation between these variables was more substantial in 
English lessons (r = 0.54) than in mathematics (r = 0.49). It means that in this research, students will be 
more engaged in the learning process if they believe they can learn. 

Furthermore, the correlation value between SRL and SE was r = 0.700, p < 0.001, signifying a 
robust positive relationship between both variables. The result differs from Setiani and Wijaya (2020), 
who reported a weak positive correlation between both variables (r = 0.262). However, the effect aligned 
with Lidiawati and Helsa (2021), who found a strong positive correlation between SRL and SE (r= 0.748). 
The strong positive correlation can happen because, according to Bond and Bedenlier (2019), cognitive 
engagement and the ability towards self-regulation are highly associated. Anjarwati and Sa’adah (2021) 
also explained that student engagement in the period of online learning is known to increase student 
participation from students’ cognitive and behavioral aspects. 


Conclusions 


This study found students’ intrapersonal factors, namely Academic Intrinsic Motivation (AIM), 
Academic Self-Efficacy (ASE), Self-Regulated Learning (SRL), and Perceived Creativity Fostering 
Teacher Behavior (P-CFTB), can determine student engagement by 66.9%, with ASE having the highest 
influence (23.99%) and P-CFTB having the lowest impact (9.78%). It also found that most participants 
belong to the high SE, P-CFTB, AIM, ASE, and moderate SRL categories. There was also a moderate 
correlation among the variables, with ASE and SE showing the most substantial relationship (r = 0.714, 
p <.001). However, P-CFTB and SE exhibited a moderate correlation between the independent and 
the dependent variables (r = 0.593, p <.001), with P-CFTB and AIM portraying the weakest relationship 
(0.468). This indicated that the strongest and weakest correlation values were found between ASE-SE 
and P-CFTB-SE, respectively. The results also showed that P-CFTB, AIM, ASE, and SRL increased SE 
among Psychology undergraduates taking online Statistics courses. 

The limitation of this study is the high-value questionnaire items, causing participants to experience 
fatigue during the fill-out process. The variables also promoted a high social desirability tendency. 
Subsequently, a delay was found between filling out the questionnaire and completing the Statistics 
courses. For example, when filling out the data instrument, students were already in semester 5, despite 
the last Statistics courses conducted in semester 3. 

According to the research, students’ intrapersonal factors in online Statistics courses significantly 
impact their level of engagement. Therefore, statistics lecturers are expected to create a learning 
atmosphere that enhances students’ intrapersonal factors, namely AIM, P-CFTB, ASE, and SRL. The 
research implies that our study can pinpoint the contribution of intrapersonal factors that affect student 
engagement, enabling statistics lecturers to give these internal factors more attention. 


Acknowledgments 


The authors are grateful to HIBAH PUT] PASCASARJANA 2022, Universitas Indonesia (NKB-297/ 
UN2.RST/HKP.05.00/2022), for financially supporting this research. 


Conflict of interests 
The authors declare no conflict of interest. 


www.ijcrsee.com 
369 


Setiamurti, A. et al. (2023). Factors affecting student engagement in psychology undergraduates studying online statistics 
courses in Indonesia, International Journal of Cognitive Research in Science, Engineering and Education (lIJCRSEE), 11(3), 
359-373. 


Author Contributions 


Conceptualization, A.S., R.M.A.S, M.N., A.A.M., F.M.M., and S.S.; methodology, A.S., R.M.A.S, 
M.N., A.A.M., F.M.M., and S.S.; formal analysis, A.S., R.M.A.S, M.N., A.A.M., F.M.M.; writing—original 
draft preparation, A.S., M.N., and A.A.M.; writing—review and editing, A.S., M.N., and A.A.M. All authors 
have read and agreed to the published version of the manuscript. 


References 


Abd-Elmotaleb, M., & Saha, S. K. (2013). The role of academic self-efficacy as a mediator variable between perceived academic 
climate and academic performance. Journal of Education and Learning, 2(3). httos://doi.org/10.5539/jel.v2n3p117 

Almarghani, E. M. & Mijatovic, |. (2017). Factors affecting student engagement in HEls - It is all about good teaching. Teaching 
in Higher Education, 22(8), 940-956, https://doi.org/10.1080/13562517.2017.1319808 

Anjarwati, R., & Sa’adah, L. (2021). Student learning engagement in the online class. EnJourMe (English Journal of Merdeka) : 
Culture, Language, and Teaching of English, 6(2), 39-49. httos://doi.org/10.26905/enjourme.v6i2.6128 

Barnard, L., Lan, W. Y., To, Y. M., Paton, V. O., & Lai, S.-L. (2009). Measuring self-regulation in online and blended learning 
environments. The Internet and Higher Education, 12(1), 1-6. https://doi.org/10.1016/.ineduc.2008.10.005 

Barnard-Brak, L., Paton, V. O., & Lan, W. Y. (2010). Profiles in self-regulated learning in the online learning environment. 
International Review of Research in Open and Distributed Learning, 11(1), 61-80. https://doi.org/10.19173/irrodl. 
v11i1.769 

Bassi, M., Steca, P., Fave, A. D., & Caprara, G. V. (2007). Academic self-efficacy beliefs and quality of experience in learning. 
Journal of Youth and Adolescence, 36(3), 301-312. https://doi.org/10.1007/s10964-006-9069-y 

Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation 
of self-report measures. Spines, 25(24), 3186-3191. https://journals.lww.com/spinejournal/citation/2000/12150/ 
quidelines_for_the_process_of_cross_cultural.14.aspx 

Bell, H., Limberg, D., Jacobson, L., & Super, J. T. (2014). Enhancing Self-Awareness Through Creative Experiential-Learning 
Play-Based Activities. Journal of Creativity in Mental Health, 9(3), 399-414. https://doi.org/10.1080/15401383.2014.8 
97926 

Bond, M., & Bedenlier, S. (2019). Facilitating student engagement through educational technology: Towards a conceptual 
framework. Journal of Interactive Media in Education, 2019(1). https://doi.org/10.5334/jime.528 

Calabrese, G., Leadbitter, D. M., Trindade, N. D. S. M. D., Jeyabalan, A., Dolton, D., & ElShaer, A. Personal tutoring scheme: 
Expectations, perceptions and factors affecting students’ engagement. Frontiers in Education, 6, 1-11. https://doi. 
org/10.3389/feduc.2021.727410 

Carpenter, T. P., & McDonald, J. (2017, May 1). Teaching (and learning) psychology statistics in an age of math anxiety. 
American Psychological Association. https://psychlearningcurve.org/psychology-statistics/ 

Cayubit, R. F. O. (2022). Why learning environment matters? An analysis on how the learning environment influences the 
academic motivation, learning strategies and engagement of college students. Learning Environments Research, 
25(2), 581-599. httos://doi.org/10.1007/s10984-021-09382-x 

Chang, D., & Chien, W. cheng. (2015). Determining the relationship between academic self-efficacy and student engagement 
by meta-analysis. Proceedings of the 2015 International Conference on Education Reform and Modern Management, 
142-145. https://doi.org/10.2991/ermm-15.2015.37 

Christenson, S., Reschly, A. L., & Wylie, C. (2012). Handbook of research on student engagement. Springer. https://link. 
springer.com/book/10.1007/978-1-4614-2018-7 

Cropley, A. J. (1997). Fostering creativity in the classroom: General principles. In M. Runco (Ed.), The creativity research 
handbook (Vol. 1, Issue Creskill, NJ: Hampton Press., pp. 83- 114). Hampton Press. http://scottbarrykaufman.com/ 
wp-content/uploads/2019/07/Runcohandbookchapter.pdf 

Czerkawski, B. C., & Lyman, E. W. (2016). An instructional design framework for fostering student engagement in online 
learning environments. TechTrends, 60(6), 532-539. https://doi.org/10.1007/s11528-016-0110-z 

Dierendonck, C., Toth-Kiraly, |., Morin, A. J. S., Kerger, S., Milmeister, P., & Poncelet, D. (2023). Testing associations 
between global and specific levels of student academic motivation and engagement in the classroom. The Journal of 
Experimental Education, 91(1), 101-124. https://doi.org/10.1080/00220973.2021.1913979 

Elias, S. M., & MacDonald, S. (2007). Using past performance, proxy efficacy, and academic self-efficacy to predict 
college performance. Journal of Applied Social Psychology, 37(11), 2518-2531. httos://doi.org/10.1111/).1559- 
1816.2007.00268.x 

Elshami, W., Taha, M. H., Abdalla, M., E., Abuzaid, M., Saravanan, C., & Al Kawas, S. (2022). Factors that affect student 
engagement in online learning in health professions education. Nurse Education Today, 110, 1-8. https://doi. 
org/10.1016/j.nedt.2021.105261 

Fan, W., & Williams, C. M. (2010). The effects of parental involvement on students’ academic self-efficacy, engagement and 
intrinsic motivation. Educational Psychology, 30(1), 53-74. httos://doi.org/10.1080/014434 10903353302 

Fatoni, Arifiati, N., Nurkhayati, E., Nurdiawati, E., Pamungkas, G., Adha, S., Purwanto, A., Julyanto, O., & Azizi, E. (2020). 
University students online learning system during Covid-19 pandemic: Advantages, constraints and solutions. 
Systematic Reviews in Pharmacy, 11(7), 570-576. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3986850 

Firmansyah, M.A. (2017). Analisis hambatan belajar mahasiswa pada mata kuliah statistika [Analysis of student learning barriers 
in statistics courses]. Jurnal Penelitian Dan Pembelajaran Matematika, 10(2), 115-127. https://doi.org/10.30870/jppm. 
v10i2.2036 

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. 
Review of Educational Research, 74(1), 59-109. httos://doi.org/10.3102/00346543074001059 


www.ijcrsee.com 
370 


Setiamurti, A. et al. (2023). Factors affecting student engagement in psychology undergraduates studying online statistics 
courses in Indonesia, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 11(3), 
359-373. 


Gettle, C. M. (2022). How personality traits and academic motivation affect engagement in synchronous online University 
courses [Undergraduate Thesis]. University of Regina. httos://instrepo-prod.cc.uregina.ca/handle/10294/14894 
Giesbers, B., Rienties, B., Tempelaar, D., & Gijselaers, W. (2013). Investigating the relations between motivation, tool use, 

participation, and performance in an e-learning course using web-videoconferencing. Computers in Human Behavior, 


29(1), 285-292. https://doi.org/10.1016/j.chb.2012.09.005 
Golder, J. (2018). Students’ perceptions on teacher behaviour at college level. International Education & Research Journal 


(IERJ), 4(6), 13-14. httos://www.academia.edu/download/65833349/05_IERJ20188551152783 Print Joydip Golder. 
df 


par 

Grégoire, J. (2016). Understanding creativity in mathematics for improving mathematical education. Journal of Cognitive 
Education and Psychology, 15(1), 24-36. httos://doi.org/10.1891/1945-8959.15.1.24 

Hafizi, M. H. M., & Kamarudin, N. (2020). Creativity in mathematics: Malaysian perspective. Universal Journal of Educational 
Research, 8(3 3C), 77-84. httos://doi.org/10.13189/ujer.2020.081609 

Hariyanto, D., Triyono, M. B., & Kohler, T. (2020). Usability evaluation of personalized adaptive e-learning system using USE 
questionnaire. Knowledge Management and E-Learning, 12(1), 85-105. https://doi.org/10.34105/.kmel.2020.12.005 

Harper, S. R., & Quaye, S. J. (2009). Student engagement in higher education: Theoretical perspectives and practical 
approaches for diverse populations. In S. R. Harper & S. J. Quaye (Eds.), Student engagement in higher education. 
Routledge. 

Hayes, A. F. (2015). An index and test of linear moderated mediation. Multivariate behavioral research, 50(1), 1-22. httos://doi. 
org/10.1080/00273171.2014.962683 

Helsa, H., & Lidiawati, K. R. (2021). Peran self-efficacy terhadap student engagement pada mahasiswa dalam pandemi 
COVID-19. Psibernetika, 14(2). htto://dx.doi.org/10.30813/psibernetika.v1412.2887 

Hu, M., Li, H., Deng, W., & Guan, H. (2016). Student engagement: One of the necessary conditions for online learning. 2016 
International Conference on Educational Innovation through Technology (EITT), 122-126. httos://doi.org/10.1109/ 
EITT.2016.31 

Huang, X. (2022). Constructing the associations between creative role identity, creative self-efficacy, and teaching for creativity 
for primary and secondary teachers. Psychology of Aesthetics, Creativity, and the Arts. https://doi.org/10.1037/ 
aca0000453 

Ifdil, |., Bariyyah, K., Dewi, A. K., & Rangka, |. B. (2019). The College Academic Self-Efficacy Scale (CASES); An Indonesian 
validation to measure the self-efficacy of students. Jurnal Kajian Bimbingan Dan Konseling, 4(4), 115-121. httos://doi. 
org/10.17977/um001v4i42019p115 

Jeffrey, B., & Craft, A. (2004). Teaching creatively and teaching for creativity: Distinctions and relationships. Educational 
Studies, 30(1), 77-87. https://doi.org/10.1080/0305569032000159750 

Kaplan, R. M., & Saccuzzo, D. P. (2017). Psychological testing: Principles, applications, and issues. Cengage Learning. 

Karwowski, M., Gralewski, J., & Szumski, G. (2015). Teachers’ Effect on Students’ Creative Self-Beliefs Is Moderated by 
Students’ Gender. Learning and Individual Differences, 44, 1-8. https://doi.org/10.1016/.lindif.2015.10.001 

Katz-Buonincontro, J., Perignat, E., & Hass, R. W. (2020). Conflicted epistemic beliefs about teaching for creativity. Thinking 
Skills and Creativity, 36(January). https://doi.org/10.1016/j.tsc.2020.100651 

Khusniyah, N. L., & Hakim, L. (2019). Efektivitas pembelajaran berbasis daring: Sebuah bukti pada pembelajaran bahasa 
Inggris [The effectiveness of online learning: An evidence on English language learning]. Jurnal Tatsqif, 17(1), 19-33. 
https://doi.org/10.20414/jtq.v17i1.667 

Kim, D. J., Bae, S. C., Choi, S. H., Kim, H. J., & Lim, W. (2019). Creative character education in mathematics for prospective 
teachers. Sustainability, 11(6), 1-16. https://doi.org/10.3390/su11061730 

Kuh, G. D. (2009). What student affairs professionals need to know about student engagement. Journal of College Student 
Development, 50(6), 683-706. httos://doi.org/10.1353/csd.0.0099 

Kuntarto, E. (2017). Keefektifan model pembelajaran daring dalam perkuliahan bahasa Indonesia di perguruan tinggi [The 
effectiveness of online learning in Indonesian language course in higher education]. Journal Indonesian Language 
Education and Literature, 3(1), 99-110. httos://doi.org/10.24235/ileal.v3i1.1820 

Lawton, S., & Taylor, L. (2020). Student perceptions of engagement in an introductory statistics course. Journal Of Statistics 
Education, 2020(1), 45-55. https://doi.org/10.1080/10691898.2019.1704201 

Lee, I. R., & Kemple, K. (2014). Preservice teachers’ personality traits and engagement in creative activities as predictors of 
their support for children’s creativity. Creativity Research Journal, 26(1), 82-94. httos://doi.org/10.1080/10400419.20 
14.873668 

Lidiawati, R. K., & Helsa, H. (2021). Pembelajaran online selama pandemi Covid-19: Bagaimana strategi pembelajaran 
mandiri dapat mempengaruhi keterlibatan siswa [Online learning during Covid-19: How self-learning strategies can 
affect student engagement]. Jurnal Psibernetika, 14(1), 1-10. https://doi.org/http://dx.doi.org/10.30813/psibernetika. 
v14i1.2570 

Linnenbrink, E. A., & Pintrich, P. R. (2003). The role of self-efficacy beliefs in student engagement and learning in the classroom. 
Reading & Writing Quarterly, 19(2), 119-137. https://doi.org/10.1080/10573560308223 

Lloyd, S. A., & Robertson, C. L. (2012). Screencast tutorials enhance student learning of statistics. Teaching of Psychology, 
39(1), 67-71. https://doi.org/10.1177/0098628311430640 

Mao, J., Chen, J., Ling, Y., & Huebner, E. S. (2020). Impact of Teachers’ Leadership on the Creative Tendencies of Students: 
The Mediating Role of Goal-orientation. Creativity Research Journal, 32(3), 228-236. https://doi.org/10.1080/104004 
19.2020.1821569 

Maroco, J., Maroco, A. L., Campos, J. A. D. B., & Fredricks, J. A. (2016). University student’s engagement: development of 
the University Student Engagement Inventory (USEI). Psicologia: Reflexao e Critica, 29(1), 21. https://doi.org/10.1186/ 
$41155-016-0042-8 

Marvianto, R.D., & Widhiarso, W. (2019). Adaptasi Academic Motivation Scale (AMS) versi Bahasa Indonesia [Adaptation of 
the Indonesian version of the Academic Motivation Scale (AMS)]. Gadjah Mada Journal of Psychology, 4(1), 87-95. 


https://doi.org/10.22146/gamajop.45785 
Maulana, H. A., & Iswari, R. D. (2020). Analisis tingkat stres mahasiswa terhadap pembelajaran daring pada mata kuliah 


www.ijcrsee.com 
371 


Setiamurti, A. et al. (2023). Factors affecting student engagement in psychology undergraduates studying online statistics 
courses in Indonesia, International Journal of Cognitive Research in Science, Engineering and Education (lIJCRSEE), 11(3), 
359-373. 


statistik bisnis di pendidikan vokasi [Analysis of student stress levels towards online learning in business statistics 
courses in vocational education]. Khazanah Pendidikan, 14(1). httos://doi.org/10.30595/jkp.v14i1.8479 

Midi, H., Sarkar, S. K., & Rana, S. (2010). Collinearity diagnostics of binary logistic regression model. Journal of Interdisciplinary 
Mathematics, 13(3), 253-267. https://doi.org/10.1080/09720502.2010.10700699 

Mustafa, J., Zayed, N. M., Islam, M.S., & Islam, S. (2018). Students’ perception towards their teachers’ behaviour: A case study 
on the undergraduate students of Daffodil International University. International Journal of Development Research, 
8(10), 23387-23392. https://www.researchgate.net/publication/329040464 

Mutiara, T., & Rifameutia, T. (2021). Adaptasi alat ukur regulasi diri dalam belajar daring [Adaptation of self-regulation 
measurement tools in online learning]. Edcomtech: Jurnal Kajian Teknologi Pendidikan, 6(2), 301-309. https://doi. 
org/10.17977/um039v6i12021p301 

Myint, K. M., & Khaing, N. N. (2020). Factors influencing academic engagement of university students: A meta-analysis 
study. Journal of the Myanmar Academy of Arts and Science, 18(9B), 185-199. https://www.researchgate.net/ 
publication/352441444 

Nunnally, J., & Bernstein, |. (1994). Psychometric theory (3" ed.). McGraw-Hill. 

Nurfitri, Y. A., & Aslamawati, Y. (2021). Pengaruh self-regulated learning terhadap student engagement pada mahasiswa 
prodi teknik informatika [The influence of self-regulated learning on student engagement in informatics engineering 
students]. Prosiding Psikologi, 7(2), 489-493. http://dx.doi.org/10.29313/.v0i0.28413 

Orr, A. M., & Kukner, J. M. (2015). Fostering a creativity mindset in content area pre-service teachers through their use of 
literacy strategies. Thinking Skills and Creativity, 16, 69-79. httos://doi.org/10.1016/j.tsc.2015.02.003 

Owen, S. V, & Froman, R. D. (1988). Development of a college academic self-efficacy scale. Educational Resources Infoemation 
Center. https://eric.ed.gow/?id=ED298158 

Osborne, J. W., & Waters, E. (2002). Four assumptions of multiple regression that researchers should always test. Practical 
Assessment, Research, and Evaluation, 8(2). httos://doi.org/10.7275/r222-hv23 

Pachler, D., Kuonath, A., & Frey, D. (2019). How transformational lecturers promote students’ engagement, creativity, and task 
performance: The mediating role of trust in lecturer and self-efficacy. Learning and Individual Differences, 69, 162-172. 
https://doi.org/10.1016/.lindif.2018.12.004 

Palaniappan, A. K. (2009). Creative teaching and its assessment. Paper Presentation, 12" UNESCO-APEID International 
Conference with the Theme “Quality Innovations for Teaching and Learning”, Impact Exhibition and Convention Center, 
Bangkok, Thailand, March 24-26, 2009, 1-15. http://eprints.um.edu.my/541/ 

Pramisjayanti, D., & Khoirunnisa, R. N. (2022). Hubungan antara self-efficacy dengan student engagement pada siswa SMP X 
kelas VIII selama masa pandemi COVID-19 [The relationship between self-efficacy and student engagement in class 
VIII students of X junior high school during the COVID-19 pandemic]. Character: Jurnal Penelitian Psikologi, 9(1). 
https://ejournal.unesa.ac.id/index.php/character/article/view/44709 

Primana, L. (2015). Pengaruh dukungan makna belajar dari dosen, motivasi intrinsik, self efficacy, dan pandangan otoritas 
sumber informasi terhadap keterlibatan belajar Mahasiswa Universitas Indonesia [The contribution of lecturer's 
meaning support in learning, intrinsic motivation, self-efficacy, and student's perceived epistemic authority, on learning 
engagement of Universitas Indonesia’s students] [Doctoral Dissertation, Universitas Indonesia]. httos://lib.ui.ac.id/ 
detail?id=20416085&lokasi=lokal 

Raviv, A., Bar-Tal, D., Raviv, A., Biran, B., & Sela, Z. (2003). Teachers’ epistemic authority: Perceptions of students and 
teachers. Social Psychology of Education, 6(1), 17-42. https://doi.org/10.1023/A:1021724727505 

Rodgers, T. (2008). Student engagement in the e-learning process and the impact on their grades. /nternational Journal of 
Cyber Society and Education Pages, 1(2), 143-156. https:/Avww.learntechlib.org/p/209167/ 

Rusman, Abd. A., & Nasution, F. (2020). Deskripsi kebahagiaan belajar mahasiswa BKI pada masa pandemi COVID-19 
[Description of BKI students’ learning happiness during the COVID-19 pandemic]. AL-IRSYAD, 10(1), 1. httos://doi. 
org/10.30829/al-irsyad.v10i1.7649 

Sa'diyah, S. H. (2021). On off kamera dan implikasinya pada perkuliahan daring [Camera on off and its implications for online 
lectures]. Jurnal Pendidikan Indonesia (Japendi), 2(9), 1593-1603. https://doi.org/10.59141/japendi.v2i09.286 

Sarjana, K., Hayati, L., & Wahidaturrahmi, W. (2020). Mathematical modelling and verbal abilities: How they determine students’ 
ability to solve mathematical word problems? Beta: Jurnal Tadris Matematika, 13(2), 117-129. httos://doi.org/10.20414/ 
betajtm.v13i2.390 

Schermelleh-Engel, K., Moosbrugger, H., & Muller, H. (2003). Evaluating the fit of structural equation models: Tests of 
significance and descriptive goodness-of-fit measures. Methods of psychological research online, 8(2), 23-74. https:// 
www.stats.ox.ac.uk/~snijders/mpr_Schermelleh.pdf 

Setiani, S., & Wijaya, E. (2020). The relationship between self-regulated learning with student engagement in college students 
who have many roles. Proceedings of the 2” Tarumanagara International Conference on the Applications of Social 
Sciences and Humanities (TICASH 2020), 307-312. httos://doi.org/10.2991/assehr.k.201209.045 

Skinner, E.A., Kindermann, T.A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection. Educational 
and Psychological Measurement, 69(3), 493-525. https://doi.org/10.1177/0013164408323233 

Soh, K. C. (2000). Indexing creativity fostering teacher behavior: A preliminary validation study. The Journal of Creative 
Behavior, 34(2), 118-134. https://doi.org/10.1002/j.2162-6057.2000.tb01205.x 

Staikopoulos, A., OKeeffe, |., Yousuf, B., Conlan, O., Walsh, E., & Wade, V. (2015). Enhancing student engagement through 
personalized motivations. 2015 IEEE 15" International Conference on Advanced Learning Technologies, 340-344. 


https://doi.org/10.1109/ICALT.2015.116 
Suminta, R. R. (2016). Kecemasan statistik ditinjau dari dukungan akademik [Statistical anxiety and academic support]. Quality, 


4(1), 120-139. http://dx.doi.org/10.21043/quality.v4i1.2117 

Ulpah, M. (2009). Belajar statistika: Mengapa dan bagaimana? [Leaning statistics: Why and how?]. INSANIA: Jurnal Pemikiran 
Alternatif Kependidikan, 14(3), 325-435. https://doi.org/10.24090/insania.v14i3.354 

Utami, R. D., & Aslamawati, Y. (2021). Pengaruh self-regulated learning terhadap student engagement pada mahasiswa prodi 
akuntansi di kota Bandung [The influence of self-regulated learning on student engagement in accounting students in 
Bandung]. Prosiding Psikologi, 7(2), 404-408. http://dx.doi.org/10.29313/.v0i0.28374 


www.ijcrsee.com 
372 


Setiamurti, A. et al. (2023). Factors affecting student engagement in psychology undergraduates studying online statistics 
courses in Indonesia, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 11(3), 
359-373. 


Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., & Vallieres, E. F. (1992). The academic motivation 
scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational and Psychological Measurement, 
52(4), 1003-1017. https://doi.org/10.1177/0013164492052004025 

van Petegem, K., Aelterman, A., Rosseel, Y., & Creemers, B. (2007). Student perception as moderator for student wellbeing. 
Social Indicators Research, 83(3), 447-463. https://doi.org/10.1007/s11205-006-9055-5 

Varatharaj, R. (2018). Assessment in the 21 century classroom: The need for teacher autonomy. International Journal of 
Research and Innovation in Social Science (IJRISS), 2(6), 105-109. httos://www.researchgate.net/profile/Ravikumar- 
Varatharaj/publication/326730856 Assessment in the 21st century classroom The nedd for teacher_autonomy/ 


links/5b7ae65592851c1e1223a7e6/Assessment-in-the-21st-century-classroom-The-nedd-for-teacher-autonomy.pdf 
Waruwu, R. B., Hao, N. P., & Hia, P. H. (2022). Analisis kemampuan pemahaman mahasiswa pada mata kuliah statistika 


di STIKES Santa Elisabeth Medan tahun 2022 [Analysis of students’ understanding ability in statistics courses at 
STIKES Santa Elisabeth Medan in 2022]. SEHATMAS: Jurnal Ilmiah Kesehatan Masyarakat, 1(3), 318-327. https:// 
doi.org/10.55123/sehatmas.v 113.653 

Warwick, J. (2008). Mathematical self-efficacy and student engagement in the mathematics classroom. MSOR Connections, 

: 31-37. https://web.archive.org/web/20170809041039id_/http://aces.shu.ac.uk/employability/resources/ 

MSOR_8331_warwickj_mathselfefficacy.pdf 

Watts, S., & Thomas, S. (2022, June 7). What is statistics in psychology? Study.Com. https://study.com/learn/lesson/statistical- 
methods-in-psychology-analysis-types-application.htm| 

Xia, Y., Hu, Y., Wu, C., Yang, L., & Lei, M. (2022). Challenges of online learning amid the COVID-19: College students’ 
perspective. Frontiers in Psychology, 13. https://doi.org/10.3389/fosyg.2022.1037311 

Zaimil, R. (2017). Analisa kesalahan mahasiswa dalam mengerjakan soal pada perkuliahan Statistika 1 FKIP Ummy Solok 
[Analysis of student errors in working on questions in the Statistics 1 Course FKIP Ummy Solok]. THEOREMS (The 
Journal of Mathematics) , 2(1), 78-85. https://ojs.fkipummy.ac.id/index.php/theorems/article/view/124 

Zhang, Y., Li, P., Zhang, Z. S., Zhang, X., & Shi, J. (2022). The Relationships of Parental Responsiveness, Teaching 
Responsiveness, and Creativity: The Mediating Role of Creative Self-Efficacy. Frontiers in Psychology, 12. https://doi. 
org/10.3389/fpsyg.2021.748321 

Zhen, R., Liu, R.-D., Ding, Y., Wang, J., Liu, Y., & Xu, L. (2017). The mediating roles of academic self-efficacy and academic 
emotions in the relation between basic psychological needs satisfaction and learning engagement among Chinese 
adolescent students. Learning and Individual Differences, 54, 210-216. httos://doi.org/10.1016/j.lindif.2017.01.017 

Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 
3-17. https://doi.org/10.1207/s15326985ep2501_2 

Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and 
future prospects. American Educational Research Journal, 45(1), 166-183. httos://doi.org/10.3102/0002831207312909 


www.ijcrsee.com 
cee 


Setiamurti, A. et al. (2023). Factors affecting student engagement in psychology undergraduates studying online statistics 
courses in Indonesia, International Journal of Cognitive Research in Science, Engineering and Education (lIJCRSEE), 11(3), 
359-373. 


www.ijcrsee.com 
374 


Bin Dawood, A., Dickinson, A., & Jones, M. (2023). Investigation of the relationship between orientation discrimination 
thresholds, autistic, and schizotypal personality traits, International Journal of Cognitive Research in Science, Engineering and 
Education (INCRSEE), 11(3), 375-387. 


Original scientific paper UDC: 

616.89-008.48:159.923.3.075(410) 
Received: March 29, 2023. 616.895.8:159.923.3.075(410) 
Revised: April 26, 2023. €) 10,23947/2334-8496-2023-11-3-375-387 


Accepted: May 11, 2023. 


® Check for updates 


Investigation of the Relationship Between Orientation Discrimination 
Thresholds, Autistic, and Schizotypal Personality Traits 


Abdullah Bin Dawood" ®, Abigail Dickinson? ©, Myles Jones’ ® 


‘Department of Psychology, King Saud University, Riyadh, Saudi Arabia, e-mail: Aaldawood1@ksu.edu.sa 
Center for Autism Research and Treatment, University of California, Semel Institute for Neuroscience, Los Angeles, United 


States of America, e-mail: abbydickinson317@gmail.com 
3Department of Psychology, University of Sheffield, Sheffield UK, e-mail: m.jones@sheffield.ac.uk 


Abstract: Imbalances in cortical excitation and inhibition (E-I) have been implicated in both autism spectrum conditions 
(ASC) and schizophrenia spectrum conditions (SSC). However, most studies investigate these clinical conditions independently, 
possibly due to the difficulty of obtaining comorbid clinical populations. As such, the current study investigated the relationship 
between performance in the orientation discrimination task “ODT” as a potential proxy for E-I balance and the autistic (as 
assessed by Autism Spectrum Quotient “AQ”) and schizotypal personality traits (as assessed by Schizotypal Personality 
Questionnaire-Brief “SPQ-BR’) in the general population. 87 healthy adult volunteers participated in the study. We found that 
high autistic personality traits are associated with enhanced performance in the oblique condition of ODT. In contrast, high 
schizotypal personality traits are associated with poor performance in the vertical condition of ODT. Such associations support 
the suggested disruption of cortical E-I balance in ASC and SSC. 

Keywords: Excitation inhibition balance, Orientation Discrimination Task, Autistic personality traits, Schizotypal personality 
traits, Visual Perception. 


Introduction 


The role of Excitation-Inhibition (E-I) balance in sensory processing has been har-nessed through 
the use of psychophysical tasks (Edden et al., 2009; Freyberg et al., 2015; Robertson et al., 2013; Snijders 
et al., 2013; Trakoshis et al., 2020; Yoon et al., 2010), see (Dickinson, Jones, et al., 2016). An advantage 
of psychophysical tasks is that the E-l balance differences inferred from them must be of ‘functional 
relevance’. Indeed, one commonly used psycho-physical task for this purpose is the visual-Orientation 
Discrimination Task (ODT) (Dickinson, Bruyns-Haylett, et al., 2016; Dickinson et al., 2014; Shaw et al., 
2019), given its well-known association with inhibitory mechanisms (Edden et al., 2009). A vast range of 
animal (Katzner et al., 2011; Leventhal et al., 2003; Sillito, 1975; Xia et al., 2013) and human research 
(Edden et al., 2009; Kurcyus et al., 2018) supports the vital role of cortical inhibition on orientation 
selectivity tuning and perceptual judgment of stimuli’s orientations due to their associations with GABA, 
the main inhibitory transmitter. For instance, animal research showed that orientation selectivity tuning 
reduced following the application of GABA antagonist (Katzner et al., 2011; Sillito, 1975) but increased 
following the application of GABA agonist (Leventhal et al., 2003; Xia et al., 2013). Similarly, enhanced 
visual orientation discrimination skills in humans are linked to higher GABA concentration levels in the 
visual cortex (Edden et al., 2009; Kurcyus et al., 2018). Together, these findings suggest that performance 
in ODT may provide a useful, al-beit indirect, indicator of cortical E-I balance in the visual cortex. 

ODT traditionally uses two different conditions, namely cardinal and oblique conditions. In the 
cardinal condition, participants judge orientation deviation from vertical or horizontal reference stimuli 
(i.e., 0/90 degrees). In the oblique condition, however, participants judge orientation deviation from 
oblique reference stimuli (i.e., 45/135 degrees). It is widely recognised that performance in the cardinal 
condition is superior to that in the oblique condition (Dickinson et al., 2015; Dickinson et al., 2014; Edden 
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et al., 2009; Orban et al., 1984; Vogels and Orban, 1985), possibly due to the larger number of cortical 
cells tuned to cardinally orientated objects compared to obliquely oriented ones (Hubel and Wiesel, 1968; 
Mansfield, 1974; Mansfield, 1974). This neural orientation selectivity has been proposed to explain the 
well-known ‘oblique effect’, which refers to performance deficiency in judging obliquely oriented stimuli 
compared to cardinally oriented ones (Appelle, 1972). 

Several studies have shown ODT performance differences in individuals with autism spectrum 
conditions (ASC) and schizophrenia spectrum conditions (SSC) compared to neurotypical (NT) 
individuals (Dickinson, Bruyns-Haylett, et al., 2016; Shaw et al., 2019). For instance, compared to NT 
individuals, those with ASC have been reported to have enhanced performance in oblique orientation 
discrimination (as indicated by lower OD thresholds) (Dickinson, Bruyns-Haylett, et al., 2016). However, 
the same enhancement was absent for the vertical condition of ODT, possibly due to the ceiling effect 
(Garin, 2014a), as vertical OD thresholds are about one-fifth of the oblique OD thresholds (Edden et al., 
2009). Additionally, compared to NT individuals, individuals with SSC have been reported to have poorer 
performance in ODT (as indicated by higher OD thresholds) in vertical and oblique ODT conditions (Shaw 
et al., 2019). Both enhanced and impaired performance in ODT has been suggested to reflect cortical 
E-| imbalance (Dickinson, Bruyns-Haylett, et al., 2016; Shaw et al., 2019). However, given the practical 
difficulty in obtaining both comorbid and non-comorbid samples for the same investigation, most studies 
have not investigated E-I balance in ASC and SSC simultaneously (Dickinson, Bruyns-Haylett, et al., 
2016; Shaw et al., 2019). This difficulty can initially be overcome by measuring the traits associated with 
each condition in the general population (Del Giudice et al., 2010; Del Giudice et al., 2014; Ford et al., 
2018; Hurst et al., 2006). 

Indeed, in addition to ODT studies in clinical samples with ASC, studies with subclinical traits of 
ASC (as measured by Autism Spectrum Quotient “AQ”) have reported that higher autistic personality traits 
in NT individuals are associated with enhanced ODT performance in the oblique condition compared to 
those with low autistic personality traits (Dickinson et al., 2015; Dickinson et al., 2014). These findings have 
been suggested to indicate increased cortical inhibition in the visual cortex of individuals with high autistic 
personality traits compared to those with low autistic personality traits (Dickinson et al., 2015; Dickinson 
et al., 2014). This suggestion has also been supported by neurophysiological measures, as individuals 
with higher autistic personality traits have higher peak gamma frequency than individuals with low autistic 
personality traits (Dickinson et al., 2015), given that variations in peak gamma frequency have been 
associated with levels of GABA concentrations in the visual cortex (Edden et al., 2009). Notwithstanding, 
such an association between peak gamma frequency and GABA concentration levels in the visual 
cortex was not replicated in a more recent study (Cousijn et al., 2014), possibly as a consequence of 
methodological differences related to sample size and the techniques used in measuring GABA (i.e., type 
of MRS sequence) (Kujala et al., 2015). 

Although previous studies have investigated the relationship between ODT performance and autistic 
personality traits (Dickinson et al., 2015; Dickinson et al., 2014), we are unaware of any study that has 
assessed ODT performance in relation to schizotypal personality traits. Given the differences between 
ASC and SSC clinical groups (i.e., enhanced and impaired, respectively) compared to NT control groups, 
we may expect poor ODT performance in individuals with higher schizotypal personality traits. However, 
there is a link between autistic and schizotypal personality traits (Dinsdale et al., 2013; Ford et al., 2017: 
Ford and Crewther, 2014; Nenadic et al., 2021; Russell-Smith et al., 2011); see (Zhou et al., 2019). Thus, 
we may expect a similar enhanced ODT performance of individuals with high autistic personality traits in 
those with higher schizotypal personality traits. 

The current study aimed to investigate the relationship between performance in ODT and both 
autistic and schizotypal personality traits in one sample, helping to disentangle these two possibilities. 
Ultimately, investigating the relationship between ODT and both autistic and schizotypal traits may provide 
a deeper understanding of the neural mechanisms underlying subclinical autistic and schizotypal traits, 
given the suggestion that ASC and SSC are two opposites on the same spectrum (Abu-Akel et al., 2018; 
Ford et al., 2017; Nenadi¢ et al., 2021; Russell-Smith et al., 2010). 


Materials and Methods 


Participants 

Eighty seven healthy volunteers from the University of Sheffield with normal or corrected to normal 
vision participated in the study. Twenty - eight were first-year Psychology students and received credits 
for participation. The rest (N=59) were recruited from the students and staff-volunteering email list of 
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the University of Sheffield and received a £7 Amazon voucher for participation in the study. Participants 
provided (written) informed consent before taking part. The study received full ethical approval from the 
Department of Psychology University of Sheffield ethics committee. 


Orientation Discrimination Task 

We used an orientation discrimination task (ODT) initially developed by (Edden et al., 2009) and 
had previously been used to measure orientation discrimination in several studies (Bin Dawood et al., 
2020; Dickinson, Bruyns-Haylett, et al., 2016; Dickinson et al., 2015; Dickinson et al., 2014). The ODT 
measures orientation discrimination thresholds using a two-alternative forced-choice adaptive staircase 
procedure (Leek, 2001; Treutwein, 1995) and was programmed in MATLAB (The MathWorks Inc., Natick, 
MA, 2000) with PsychToolbox (Brainard, 1997). 

In each trial of ODT, two circular black and white gratings (diameter 4°; spatial frequency three 
cycles/degree; contrast 99 %; mean luminance 83 cd/m2) were sequentially presented for 350 ms with 
fixation of 500 ms between these two gratings, (Figure 1). The first presented grating is regarded as the 
reference grating, whereas the second presented grating is regarded as the target grating. Based on the 
orientation of the reference grating, the ODT comprised vertical and oblique conditions. In the vertical 
condition, the reference grating was orientated at 0°. In the oblique condition, however, the reference 
grating was orientated at 45°. Each condition has two staircases based on the target grating’s rotation 
direction (clockwise, anti-clockwise). The staircases implemented one-up three-down procedures, 
converging on 79% accuracy (Leek, 2001). For each staircase, the target grating was initially presented 5 
degrees away from the reference grating with an initial step size of 1 degree, decreasing 75% after each 
reversal. 


Target grating 
oriented at 55° 


Reference grating 
oriented at 45° 


Figure 1. Schematic diagram of the orientation discrimination task. This figure is reprinted with 
permission from (Dickinson et al., 2014). 


Participants were asked to sit comfortably on a chair 57 cm away from the monitor in a dimly lit 
room. A black circular aperture covered the monitor to eliminate any external orientation cues provided by 
the monitor edges. Participants were asked to judge whether the target grating had been tilted clockwise 
or anti-clockwise compared to the reference grating using the right and left arrow keys. Participants started 
with a practice run of 10 trials for each of the four staircases. Participants then completed the experimental 
run consisting of 140 trials for each staircase. The task finished when each staircase had reached eight 
reversals and would be terminated if 140 trails were hit for each staircase. 

To calculate OD thresholds, we used the final six reversals from each staircase after discarding the 
initial two reversals as practice trials. The thresholds of vertical and oblique conditions were calculated 
separately by averaging each condition’s right (clockwise) and left (anti-clockwise) staircases. 


Questionnaires 

To assess autistic and schizotypal personality traits, we used the Autism Spectrum Quotient (AQ) 
(Baron-Cohen et al., 2001) and the Schizotypal Personality Questionnaire-Brief Revised (SPQ-BR) 
(Cohen et al., 2010). Both AQ and SPQ-BR scales were administered online using Qualtrics (httos://www. 


qualtrics.com). 
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Autism Spectrum Quotient (AQ) 

The Autism Spectrum Quotient (AQ) is a 50-item self-report Quotient for measuring autistic 
personality traits in the general population (Baron-Cohen et al., 2001). The items are scored on a 4-point 
Likert scale with response options ranging from ‘definitely agree’ to ‘definitely disagree’. The responses are 
collapsed into a dichotomous scoring scheme as responses with an endorsement of autistic personality 
traits are scored 1 while the opposite responses are scored 0. The five AQ subscales assess imagination, 
attention to detail, communication, attention switching, and social skills subscales. The total score of AQ 
ranges from 0-50 for the complete inventory, with a higher score indicative of a higher level of autistic 
personality traits. 


Schizotypal Personality Questionnaire-Brief Revised 

The Schizotypal Personality Questionnaire-Brief Revised (SPQ-BR) is a 32-item self-report 
questionnaire for measuring schizotypal personality traits (Cohen et al., 2010). The items are scored on a 
5-point Likert scale with response options ranging from ‘strongly disagree’ to ‘strongly agree’. The SPQ-BR 
consists of positive (i.e., “Cognitive-Perceptual”), negative (i.e., “Interpersonal”), and disorganised (i.e., 
“Disorganized”) subscales, with a total score ranging from 0 to 128 for the complete inventory whereby a 
higher score is indicative of a higher level of schizotypal traits. 


Exclusion Criteria 

Three exclusion criteria were applied. One criterion was failing to reach six staircase reversals in 
any condition of ODT. Another criterion was having OD thresholds with more than two standard deviations 
(SD) away from the average (mean). The last criterion was failing to complete the survey. Data from 16 
participants were excluded. Two participants failed to reach six staircase reversals in any ODT condition. 
Also, the OD thresholds of six participants in any condition of ODT were more than 2 SD away from the 
average. Additionally, eight participants did not complete the online survey, Tables 1 and 2. 


Table 1. Illustrates the demographic information of 71 participants (46 female, 25 male) whose 
data were included in the analyses after applying the exclusion criteria stated earlier. 


Variable Mean Standard Deviation 

Age 22.42 4.82 

Vertical Orientation Discrimination Threshold (in degrees) 1.76 1.06 

Oblique Orientation Discrimination Threshold (in degrees) 7.80 2.38 

Autism Spectrum Quotient (AQ)Score = —its—‘“‘<is;~s*é*é=és«sBSBC‘<‘(‘<#(#(‘( (CS OTB! 
Schizotypal Personality Questionnaire-Brief Revised (SPQ-BR) 85.11 16.21 


Score 
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Table 2. Illustrates each participant's demographic information after applying the exclusion criteria 
stated earlier 
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Results 


Data from 71 participants (46 female, 25 male; mean age 22.42; SD= 4.82; age range = 18-37) were 
used to examine the relationship between ODT performance, autistic personality traits, and schizotypal 
personality traits. The mean threshold of the vertical condition of ODT was 1.76° with a standard deviation 
of 1.06°, while the mean threshold of the oblique condition of ODT was 7.80° with a standard deviation 
of 2.38°. Paired sample t-test analysis showed a statistically significant difference between vertical and 
oblique ODT performance (t (70) =-24.88, p<.0001), Figure 2. 


12 
EK 


Orientation Discrimination Threshold (degrees) 


0 


Vertical Condition Oblique Condition 


Figure 2. Mean orientation discrimination thresholds for the vertical and oblique conditions of the 
orientation discrimination task. Error bars represent standard deviation. ***p < 0.0001. 


AQ scores ranged from 6 to 37 (mean=18.83, SD=6.78), and SBQ-BR scores ranged from 56 to 
127 (Mean=85.11, SD=16.21). The internal consistency analysis of both AQ and SPQ-BR scales revealed 
high reliability, with Cronbach's alpha of .80 for AQ and .85 for SPQ-BR. 

To examine the relationship between performance in ODT, autistic personality traits, and schizotypal 
personality traits, a Pearson correlation analysis was performed using the following variables: vertical OD 
thresholds, oblique OD thresholds, AQ scores, and SPQ-BR scores. 

We found a statistically significant positive correlation between vertical OD thresholds and SPQ- 
BR scores, (r= .304, n=71, p=.010). Higher vertical OD thresholds (worse task performance) correlated 
with higher SPQ-BR scores, (Figure 3(a)). Also, we found a statistically significant negative correlation 
between oblique OD thresholds and AQ scores, (r= -.241, n=71, p=.043). Lower oblique OD thresholds 
(better task performance) correlated with higher AQ scores, (Figure 3(b)). However, we found no significant 
correlation between vertical OD thresholds and AQ scores (r=.052, n=71, p=.33), nor was it between 
oblique OD thresholds and SPQ-BR scores (r=-.036, n=71, p=.77). 
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Figure 3. (a) A correlation between vertical orientation discrimination thresholds and SPQ-BR 
scores (r= .304, n=71 p=.010), and (b) A correlation between oblique orientation discrimination thresholds 
and AQ scores (r= -.241, n=71 p=.043). 


Additionally, the result showed that AQ scores positively correlated with SPQ-BR scores (r= .455, 
n=71, p< .0001), (Figure 4). 
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Figure 4. A correlation between AQ scores and SPQ-BR scores (r= .455, n=71, p< .0001). 


Furthermore, two partial correlation analyses were performed, given the correlational relationship 
between AQ and SPQ-BR. 

The first partial correlation analysis was to assess the relationship between vertical OD thresholds 
and SPQ-BR scores while controlling for AQ scores. The result revealed a statistically significant positive 
correlation between vertical OD thresholds and SPQ-BR scores when the AQ scores were controlled for 
(r=.316 n=71, p=.008). 

The second partial correlation analysis assessed the relationship between oblique OD thresholds 
and AQ scores while controlling for the SPQ-BR scores. The result revealed a statistically negative 
correlation between oblique OD thresholds and AQ scores when the SPQ-BR scores were controlled for 
(r=-.253, n=71, p=.035). 

In addition, to confirm that the associations between OD thresholds and scores in personality traits 
were not driven by other possible factors such as age, which has been linked to cortical E-| balance in 
previous studies (Abuleil, McCulloch and Thompson, 2019; King etal., 2020; Petitet etal., 2021; Pitchaiuthu 
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et al., 2017), further correlation analyses were performed. The results indicated no statistically significant 
correlation between age and both OD thresholds and personality traits, with p-values greater than .05, 
Table 3. A possible explanation for the insignificant associations between age and other variables may be 
due to the narrow range of age (18- 37 years). 


Table 3. illustrates the correlations between age and both OD thresholds and scores 
in personality traits. 


Vertical OD Obli OD 
pias see AQ Scores SPQ Scores 
Thresholds (°) Thresholds (°) 
AGE Pearson's r 0.003 -0.121 0.142 0.002 
p-value 0.981 0.315 0.236 0.988 
Discussion 


The current study aimed to replicate previous studies examining the relationship between 
performance in ODT and autistic personality traits (Dickinson et al., 2015; Dickinson et al., 2014) finding 
that higher levels of autistic personality traits were associated with lower thresholds in the oblique, but not 
vertical, condition of ODT. In the same group of participants, we found that schizotypal personality traits 
were linked to ODT performance in the vertical but not in the oblique condition, partially consistent with 
the increased OD thresholds that have been found in a clinical sample of participants with SSC (Dickinson 
et al., 2015; Dickinson et al., 2014), finding that higher levels of autistic traits were associated with lower 
thresholds in the oblique, but not vertical, condition of ODT. In the same group of participants, we found 
that schizotypal personality traits were linked to ODT performance in the vertical but not in the oblique 
condition, partially consistent with the increased thresholds that have been found in a clinical sample of 
participants with SSC (Shaw et al., 2019). Specifically, individuals with higher schizotypal personality traits 
showed increased OD thresholds (i.e., worse performance) in the vertical condition. Finally, we found that 
higher autistic personality traits are associated with higher schizotypal personality traits, consistent with 
previous findings (Dinsdale et al., 2013; Ford et al., 2017; Ford and Crewther, 2014; Russell-Smith et al., 
2011). 


As expected based on the hypothesis of increased inhibition in the ASC (Bonnel et al., 2010; 
Bonnel et al., 2003; Dickinson, Bruyns-Haylett, et al., 2016; Dickinson et al., 2014), we found that high 
autistic personality traits (indexed by high AQ scores) are associated with enhanced performance in 
ODT (indexed by low OD thresholds). In line with previous results (Dickinson et al., 2015; Dickinson et 
al., 2014), this association was condition-specific as it was only observed in the oblique ODT condition. 
A possible explanation for the lack of a significant relationship between autistic personality traits and 
OD thresholds in the vertical condition has been attributed to the ceiling effect (Dickinson et al., 2014). 
Compared to OD thresholds in the oblique condition (M=7.81°, SD=2.42°), OD thresholds in the vertical 
condition were very low (M=1.78°, SD=1.05°), possibly preventing any potential performance differences 
from being detectable (Garin, 2014a). Similar to this orientation-specific association between performance 
in ODT and autistic personality traits, Edden et al. (2009) found that GABA concentration level in the visual 
cortex was only significantly associated with performance in the oblique condition of ODT but not with 
that of vertical condition, possibly due to ceiling effect. Our finding of an association between high autistic 
personality traits and enhanced performance in the oblique condition of ODT supports the suggestion of 
increased cortical inhibition in the visual cortex of individuals with high autistic personality traits (Dickinson 
et al., 2015; Dickinson et al., 2014). 

Consistent with the hypothesis of disrupted E-I balance in the SSC (Abbasi et al., 2023: Anticevic 
and Lisman, 2017; Lisman, 2012), the current study found an association between high schizotypal 
personality traits (indexed by higher SPQ-BR score) and poor performance in ODT (indexed by high OD 
thresholds). However, this association was only statistically significant for the vertical condition of ODT 
but not for the oblique condition. Such poor performance in the vertical condition of ODT may reflect 
decreased inhibition in the visual cortex, which has been suggested to be associated with SSC (Goto et 
al., 2009; Rokem et al., 2011; Yoon et al., 2010; Yoon et al., 2009). For instance, reduced cortical inhibition 
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is associated with a broader orientation tuning curve of visual cortical neurons of individuals with SSC 
(Rokem et al., 2011). As such, the lack of observing any significant relationship between high schizotypal 
personality traits and performance in the oblique condition of ODT might be due to the so-called ‘floor 
effect’ (Garin, 2014b). This is because the visual orientation-tuning curve is initially broader for oblique 
orientations than the vertical ones (Ringach, 1998), possibly limiting any further performance reduction 
from being detectable. 

Additionally, our finding of an association between high schizotypal personality traits and poorer 
ODT performance is consistent with that of (Shaw et al., 2019) finding that individuals with SSC had 
poorer ODT performance than NT individuals. However, it should be noted that (Shaw et al., 2019) found 
that individuals with SSC had higher OD thresholds in both vertical and oblique conditions. Such partial 
inconsistency could be related to the type of population. For example, while the study of (Shaw et al., 
2019) included a sample of clinically diagnosed individuals with SSC, our current study included a sample 
of NT individuals with subclinical traits. 

In line with previous studies (Dinsdale et al., 2013; Ford et al., 2017; Ford and Crewther, 2014; 
Russell-Smith et al., 2011), we also found a significant association between autistic and schizotypal 
personality traits as high AQ scores are associated with high SPQ-BR scores. This is expected based 
on previous studies showing an association between ASC and SSC (Burbach and van der Zwaag, 2009: 
Chisholm et al., 2015; King and Lord, 2011; Pilowsky et al., 2000; Volkmar and Cohen, 1991; Wood, 
2017). For instance, a systematic review and meta-analysis study examining the association between 
ASC and SSC reported a 3.4% to 52% prevalence rate of ASC in individuals with SSC (Chisholm et 
al., 2015). ASC and SSC have been characterised by defects in social, cognitive, and communication 
functions (Pilowsky et al., 2000). Also, ASC and SSC have been suggested to share the same pathogenic 
mechanism (Burbach and van der Zwaag, 2009). Additionally, our finding is consistent with previous 
studies investigating the relationship between autistic personality traits and schizotypal personality 
traits (Dinsdale et al., 2013; Ford et al., 2017; Ford and Crewther, 2014; Russell-Smith et al., 2011). For 
instance, autistic personality traits and schizotypal personality traits of non-clinical participants were found 
to be positively associated with each other (Dinsdale et al., 2013). 

As previously reported (Dinsdale et al., 2013; Ford et al., 2017; Ford and Crewther, 2014; Nenadi¢ 
et al., 2021; Russell-Smith et al., 2011), AQ scores and SPQ-BR scores were correlated in the current 
study. Given their association, the question arose as to how AQ scores and SPQ-BR scores could correlate 
differentially with ODT performance. As is typical, the correlation between AQ scores and SPQ-BR scores 
is below 1 in the current study (r=0.455), allowing the possibility for the scores of these two inventories 
to independently and differentially correlate with ODT performance. To further investigate this possibility, 
partial correlations were used to demonstrate that after controlling for AQ scores, the correlation between 
SPQ-BR scores and vertical OD thresholds (r=.316) was similar to the zero-order correlation (r=.304). 
Likewise, after controlling for SPQ-BR , the correlation between AQ scores and oblique OD thresholds 
(r=-.253) was similar to zero-order correlation (r=-.241). This suggests that despite their association, AQ 
scores and SPQ-BR correlate independently and differentially with ODT performance. Indeed, similar 
findings have been found for global versus local processing (Russell-Smith et al., 2010). Despite the 
correlation between autistic and schizotypal traits, individuals with high autistic personality traits had an 
advantage or preference for local versus global processing than those with low autistic personality traits 
(Russell-Smith et al., 2010). However, individuals with high schizotypal personality traits were found to 
have an advantage or preference for global versus local processing than those with low schizotypal 
personality traits (Russell-Smith et al., 2010). Our divergent pattern of findings regarding ODT performance 
within the same sample is interesting. For instance, while ASC and SSC have been linked to enhanced 
and impaired ODT performance, respectively (Dickinson, Bruyns-Haylett, et al., 2016; Shaw et al., 2019), 
we show that these relationships remain present when studying these two types of traits within the same 
sample. Our findings may contribute unique mechanistic insight regarding E-I disruptions that cannot be 
inferred when studying autistic and schizotypal samples independently. 


Conclusion 


The current study extended previous studies examining the relationship between performance in 
ODT and autistic personality traits by examining schizotypal personality traits in the same sample. Similar 
to previous studies (Dickinson et al., 2015; Dickinson et al., 2014), we found that higher levels of autistic 
personality traits are associated with enhanced performance in the oblique condition of ODT. Additionally, 
we found that high schizotypal personality traits are associated with poorer performance in the vertical 
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condition of ODT, as expected based on the findings of (Shaw et al., 2019). Finally, we replicated the 
previously demonstrated relationship between autistic and schizotypal personality traits (Dinsdale et al., 
2013; Ford et al., 2017). Future studies should examine such associations in different cultural contexts, 
given the critical role of culture on cognition and mental health diagnoses (Banerjee, 2012; Kim, 2012; 
Phillips, 2019). 
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Abstract: In the contemporary digital era, social media platforms have a big influence on students lives. They use these 
platforms for self-expression, opinion sharing, and experience reporting (writing or sharing videos or photos about personal 
experiences) in addition to social interaction. Education professionals and academics may get valuable insights into students’ 
thoughts, sentiments, interests, academic success, and even personalities by studying their writing on social media. We can 
improve our teaching, enhance students’ social and emotional development, and create a more engaging learning environment 
if we have a better knowledge of the student. The purpose of this study is to ascertain whether or not students interact with 
classmates and other participants in learning platforms in a way that accurately represents their personalities. Data from a sample 
of students at Abdelmalek Essaadi University of Tetouan were collected from various social media learning environments for 
the experimental investigation presented in this work, and Symanto Al-based personality tool was used to assess the data. The 
Big Five Questionnaire was then utilized to assess the personalities of the same students, and the findings were compared to 
the personality traits discovered by the Al-based approach. The study has shown that the Al based tool has correctly predicted 
the personality traits of 7 students out of 10 with a correlation of about 0,9 which means that social media-based learning 
environments can be used by institutions to understand the personality of the student. This paper also gives recommendations 
about data for obtaining good quality in personality prediction. 

Keywords: FFM personalities, social media learning environment, Moroccan dialect text. 


Introduction 


Student written text plays a special and dynamic role in learning environments that uses social 
media. Actually, students use social media platforms like (Instagram, WhatsApp and Twitter) more and 
more, and there are many advantages of using an online learning environment for educational reasons 
(Christine Greenhow et al., 2019). Those platforms enable informal and spontaneous conversation 
between students. Therefore, they frequently post variety of content, and they receive in return quick 
feedback and interactions with other students and teachers. Among the advantage of this trend, these 
interactions encourages participation and teamwork in students, which can improve their learning process 
(Josué et al., 2023). Moreover, unlike standard academic writing, students may present ideas, queries, 
or observations in a less structured manner. As a result, this informal setting may foster individualism, 
creativity, and the exchange of different perspectives (Eysenck, 1994). 

Students’ writing in social media often reflects their ideas, feelings, interests, and ways of interacting, 
which can provide interesting insights about them. Thus, based on these writings, it is possible to 
determine their personalities on social media using a variety of methods, including: content and interests, 
language and communication style, frequency and consistency, tone and emotions, interaction patterns, 
etc... (Rahman et al., 2019). 

To understand individual's personality many approaches and theories are used. The following list 
presents the most well-known models: 

* The Myers-Briggs Type Indicator (MBTI) (Pittenger, 1993; Tlili et al., 2016): categorizes persons 
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into 16 personality types based on four dichotomies which are extraversion/introversion, sensing/intuition, 
thinking/feeling, and judging/perceiving. Evry combination of these caracteristics results in a distinct 
personality type, such as INFP (Introverted, Intuitive, Feeling, Perceiving) or ESTJ (Extraverted, Sensing, 
Thinking, Judging). 

* The Big Five (BF) Personality Traits (Caprara et al., 1993; Eysenck, 1994): Also known as the 
Five Factor Model (FFM). According to this Model, there are five different dimensions of personality: 
Agreeableness, Conscientiousness, Extraversion, Openness, and Neuroticism (Utami, Maharani and 
Atastina, 2021). It is frequently employed in organizational and behavioral studies as well as psychology 
research, offers a thorough framework for evaluating and characterizing personality traits. It is considered 
that these characteristics sum up the fundamental elements of human behavior and personality. 

* Freudian Personality Structure (Bronfenbrenner, 1951; Zhang, 2020): The famous Austrian 
psychotherapist Sigmund Freud developed a theory of personality structure known as the Freudian 
Personality Structure. According to Freud, there are three primary parts of the human mind (Id, Ego and 
SuperEgo). 

* Hans Eysenck’s model (Eysenck, 1991, 1981): The prominent psychologist Hans Eysenck 
developed a widely known three-dimensional model of personality: Neuroticism/Emotional Stability, 
Extraversion/Introversion, and Psychoticism. The field of personality psychology has been greatly 
impacted by Eysenck’s work, and his model has been widely applied in studies and personality evaluation. 

* DISC (Dominance, influence, Steadiness and Conscientiousness) personality Model (Sugerman, 
2009; Utami et al., 2022): Another psychological theory for understanding and classifying human behavior 
in diverse contexts is the DISC Personality Model. It categorizes people into four main personality traits, 
denoted by a different letter in the acronym DISC. This model is frequently applied in work environments 
and interpersonal interactions in order to foster better understanding, cooperation, and communication 
among people with diverse personality types. 

This work considers the Big Five model as a method of analyzing the personality of students out of 
all the models previously provided since it is the most widely used model and particularly because the Al 
algorithm for personality detection from text is built on it. 

Through the Automated Text-Based Personality Assessment (ATBPA) (Gjurkovic, Vukojevic and 
Snajder, 2022), artificial intelligence (Al) may predict personality from text by using well-established 
psychological models. These latter can determine a person’s personality traits from written content 
through analyzing the writing styles, linguistic patterns, word choices, etc.... (Christian et al., 2021). The 
Al models are trained using machine learning algorithms including text classification and natural language 
processing. The training of the model uses mainly the annotated data from a dataset. Therefore, the 
model acquires the ability to identify patterns and connections between linguistic features and personality 
traits (Gjurkovié, Vukojevié and Snajder, 2022). 

The main goal of this paper is to use one of these ATBPA tools to identify students’ personalities 
based on their writing in the Moroccan dialect in social media learning environments. For this study, 
we have chosen Symanto APIs as a tool. To achieve our goal, we have gathered data from students in 
various social learning environments (Instagram, Twitter, WhatsApp and Google chat). This data has been 
preprocessed by removing irrelevant information and then translating it into English. Subsequently, it has 
been processed by the Al-based personality algorithm to predict students’ personality traits. Finally, the 
students’ personality predictions obtained by the algorithm were compared to the Big Five Questionnaire 
results that were gathered from the same students. 

The following section presents a summary of the literature on the use of social media learning 
environment. In section 3, we explain the methodology and the applied data processing approach. 
Subsequently, the experiment’s findings are presented in Part 4 followed by an analysis of the results 
and a discussion in Section 5. Finally, the paper ends with a conclusion that summarizes the work and 
presents the implications for further research. 


Social media and education 

The term “social media” refers to a modern phenomenon that includes both mobile interaction and 
web-based communication with internet users via web applications (Wickramanayake and Muhammad 
Jika, 2018). Thanks to how convenient it is to access these applications, the majority of people utilize 
social media for a variety of purposes, including recounting experiences, communicating, and sharing 
stories from their everyday lives. In the case of students, the development of Web 2.0 and the emergence 
of Web 3.0 have enabled students to produce content, exchange ideas, and share knowledge. This 
development is definitely igniting a revolution in the world of education (Namaziandost and Nasri, 2019). 

There are now numerous social media learning environments which are frequently utilized by our 
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students. Among these, the most well-liked platforms include WhatsApp, Instagram, Facebook, Wiki, 
Skype, YouTube, LinkedIn, Blogs, Twitter and Google Chat,... (Swaminathan, Harish and Cherian, 2013). 
These platforms, according to Lim et al., can be grouped into seven categories: media sharing, text-based, 
social networking, virtual world and games, synchronous communications, conferencing applications and 
mashups, and mobile-based application (See Yin Lim et al., 2014). The top 17 social media learning 
environments in terms of monthly active users are listed in Table 1. 


Table 1 
The top 17 social media learning platforms (Barrot, 2022). 
Social SS Initial— Monthly a 
. / | ' Owner Country of origin 
media _ felease —_active users 
Facebook 2004 +~—- 2603000000 Facebook, Inc. United States 
— WhatsApp = 2009 2000000000 i it” Facebook, Inc. United States 
YouTube —«- 2005 ooo tt” Google United States 
WeChat 2011 ~—~—~—«:1203000000 : Tencen China 
7 Instagram ai 2010 00000 - Facebook, Inc. United States 
TikTok 2016 ~=~—« 800000000 : ByteDance China 
QQ 1999 694000000 Tencen China 
Linkedin 2002 660000000 Mi United Stat 
SinaWeibo 2009 550000000 — PanWeibo st - 
Reddit 2005 430000000 U 
Kuaishou 2011 400000000 Bijing Kuaishoun Technology Co., Ltd. ~~ China 
Tumbir 2007 400000000 Automattic United States. 
Snapchat 2011 397000000 | Snap, Inc. United States 
Pinterest 2009 367000000 Pinterest, Inc. United States 
Twitter 2006 326000000 ‘Twitter, Inc. United States 
Skype 2008 300000000 Skype Technolgies United States 
MySpace 2008 50600000 Meredith Corporation United States 


Materials and Methods 


This study focuses on how interactions in learning contexts reveal students’ personalities. In the 
previous section, we discussed the most common social media environments that are frequently visited 
by students and how they use them to interact about learning issues. On the other hand, there is a set of 
Al-based tools that can be used to detect the personality of people. These tools can be used either using 
Application Programming Interfaces (APIs) or Graphical User Interfaces (GUIs). In this context, we have 
gathered data from students in the classroom and we have used ‘Symanto’ (https://www.symanto.com/) 
as one of these Al-powered tools to evaluate their personalities. 

Detecting the personality based on the tool may not be enough to confirm that it is really the 
personality of the learner. Therefore, to assess the accuracy of these latter, we asked the same students 
who participated in the experiment to answer a Big Five Questionnaire test. The purpose is to compare 
test results to personality traits predicted by Al. As a final goal, we will then be able to investigate how 
learning environments may be used to understand students’ personalities. The general steps of the 


experiment are shown in Figure 1. 
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Figure 1. Summary of the student personality comparison and detection technique 
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Data collection 

This research involved ten students of Computer Science Engineering who were enrolled at the 
National School of Applied Sciences of Tetouan at the Abdelmalek Essaadi University. We gathered their 
text expressions in different contexts and from multiple social media platforms. Mainly text captured from 
comments on publications (courses, labs, exercises solutions...), discussions and publications posted 
by students themselves. The text was integrated and translated before being analyzed by the Machine 
Learning model. Meanwhile, the same students were asked to respond to the Big Five Questionnaire test. 

From numerous social media platforms, information on all target students was collected. The 
targeted platforms were chosen depending on the data that is readily available for each student. For 
example, concerning the first student (see Table 2) we gathered 51 samples from Whatsapp, 41 samples 
from Instagram, 6 samples from Twitter and 5 samples from Google Chat. The quantity of data samples 
that were collected for each student in each social media platform is displayed in the Table 2. 


Table 2 
Sample of data for each student per each platform included in the study 


Student Instagram —‘ Twitter WhatsApp Google Chat total by student 


Student 1 44 6 51 5 103 
Student 2 52 22 77 11 162 
Student 3 38 15 42 8 103 
Student 4 62 3 33 3 101 
Student 5 23 2 68 15 108 
Student 6 30 23 At 9 103 
Student 7 13 4 56 5 78 
Student 8 18 29 39 12 98 
Student 9 78 12 74 22 186 
Student 10 33 21 52 13 119 
Total 388 137 533 103 1161 
Average 38,8 13,7 53,3 10,3 116,1 


As shown in Table 2, the number of collected data sample is not equivalent comparing different 
platforms to each other’s. This is due to the fact that some platforms are more often used by students 
than others (WhatsApp and Instagram for example). The total number of data gathered is 1161, with an 
average of 116,1 samples for each student. The data was stored in CSV files, with each line containing 
the student's text samples arranged by their originating environment. To make sure that the content of the 
students’ discussions was obvious and comprehensible and that the process of organizing the data was 
completed without errors, all data was reviewed as well as we ensured the samples belonged to the right 
students. 


Data selection/preprocessing 

In this step, we have considered data of all students taking part in the study as an adequate number 
of samples was collected for each participant. In order to prevent utilizing unrepresentative samples, they 
had previously been chosen using a variety of estimated characteristics. 

For data cleaning, we removed some ilnsignificant data which made up less than 2.8% of the 
overall data. The data has been transformed before being processed due to Moroccan students” use of 
their native dialect (“darija’) in their writings. The data transformation procedure was a very challenging 
step. The text could not be used in its original form since the NLP (Natural Language Processing) Model 
does not support the Moroccan dialect. To address this issue, we translated the content into English as 
it is a language that the model can comprehend. Therefore, we have carried out a full English translation 
of the content (see examples in Table 3). The next stage was to incorporate all the student samples, 
regardless of the environment type. Subsequently, all student data were collected in one entry of the 
Model because we needed to predict each student’s personality separately. 
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Table 3 
Text language standardization to English 
Sentence in Moroccan dialect English translation 


the teacher informs you that this Sunday is the deadline to 


Yalkom Iprof nhar I7ad adir ajal dyal les projets rendu ; 
send your projects 


Lcours d8ada rah annulé tomorrow's class is canceled 

BBit les solutions dyal tepeyat | need practical works solutions 

Darni rassi 8adi nmxi | have a headache; I’m going to leave 
Soya 3afak xof m3aya l’erreur fin kayna Bro, help me find the mistake 

Walo dak logiciel mab8ach yt2anstala leya unfortunately, the software is not installed 
Nari majibtx mzyan Yafarta / alas, | did’t get a good mark 


Symanto: personality prediction tool 

To identify and analyze personalities from the text, there are several Al-based solutions. Many of 
them are a paid service for business use and can be accessed only for evaluation purpose. However, 
there are some other applications that are for free use in research domain. In this work we use the free 
evaluation API of Symanto which is a tool that provides companies with insights from customer data. 
Symanto is also provided as an API on RapidAPI (https://rapidapi.com/). 

In this experiment, we used the RapidAPI web interface to send student text, which resulted in an 
HTTP POST request to the API. The web service response contains predictions made by the Al Model 
presented in the .json data format. This last contains the probability of each personality trait based on the 
Big Five, as well as multiple subclasses of each personality trait. Only predicted values of the first level of 
the Big Five personality traits were considered in this study. 


Results 


The preliminary results of this work are the outcomes originated from examining student writing 
using Symanto Al-based tool to detect personality. The results include the probability of each personality 
trait according to the Big Five Model which is displayed in Table 4. 


Table 4 
Students’ personality detected by Symanto 


Extraversion Openness Conscientiousness Neuroticism Agreeableness 


Student 4 0,34 0,18 0,45 0.41 0,64 
Student 2 0,39 0,42 0,47 0,28 0,71 
Student 3 0,64 0,58 071 0,23 0,59 
Student 4 0,45 0,53 0,73 0,44 0,66 
Student 5 0,22 0,37 0,69 0,55 0,28 
Student 6 0,42 0,43 0,73 0,52 0,61 
Student 7 018 065 074 0,57 0,23 
Student 8 0,45 0,32 0,45 0,37 0,77 
Student 9 0,23 0,46 04 0,29 0,73 

Student 10 0,54 0,66 0,73 0,36 071 
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The same students were also requested to complete a personality questionnaire based on the Big 
Five Model, as was previously mentioned. The results of this personality test are shown in Table 5. 


Table 5 
Students’ personality detected based on the BF Questionnaire. 


Extraversion Openness  Conscientiousness  Neuroticism  Agreeableness 


Student 1 03 02 04 04 0.6 
Stuent2 =o (<testi‘<ié‘éiSC(‘(<;sé‘=étw!!!ClClOGCOS 
Student 3 07 06 08 03 06 
Student 4 05 05 0.8 04 7 
Student 6 04 04 08 06 0.5 
Students =805 O03 O65 oO4 o8 | 
Student 9 03 04 0,3 03 0,7 
Student 06 07 0.6 03 08 


Representing personality traits detected by the Al based tool and those assumed by the 
Questionnaire in a radar chart for each student can assist in comparing their results. The following figures 
illustrate the radar charts buit based on the comparison between the results of the tables presented 
previously (see Figure 2). 


Student 1 Student 2 
—SYMANTO QUESTIONNAIRE = SYMANTO. (QUESTIONNAIRE 
ctraversion 


0,6 
0,4 


Student 5 Student 6 


——SYMANTO = === QUESTIONNAIRE 


neuroticism ~~ conscientiousness 


Student 8 


—sYMANTO QUESTIONNAIRE 


Extraversion Estavestion 


agreeableness openness agreeableness « 
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Student 9 Student 10 


——SYMANTO QUESTIONNAIRE —SYMANTO QUESTIONNAIRE 


Extraversion 
O8 
06 
0s 
o4 v1 
agreeableness ——5s- openness 0,3 
8 Nees Pp agreeableness ad openness 
. o1 01 


neuroticism conscientiousness neuroticism conscientiousness 


Figure 2. Radar charts of personality predictions for each student 


The above-mentioned data demonstrates that diverse personalities predicted by the Al based tool 
do not always produce similar results based on the Questionnaire. Some students’ predictions (students 
1, 3, 4, and 8) are quite accurate. However, there is a huge difference between predicted personality of 
three students compared to their personality detected by the BF Questionnaire (Student 2, Student 5 and 
Student 7). 

Therefore, in order to evaluate results more precisely, we have decided to use metrics that can give 
more insights from the results. Bellow we present all the metrics used: 

* HPT (Highest personality traits): is when a student’s highest personality trait value from the Al- 
based tool and his highest personality trait value from the Questionnaire are same, this setting is set 
to true. This metric will give us a rate that describes how much the model predicts the most dominant 
personality trait. 

* LPT (Lowest personality traits): is when a student’s lowest personality trait value from the Al- 
based tool and his lowest personality trait value from the Questionnaire are the same, this setting is set to 
true. This metric will give us a rate that describes how much the model predicts the weakest personality 
trait in a person. 

* ME (Mean error): is a measurement of the average difference between the value of a personality 
characteristic predicted by an Al-based tool and the value supplied by a Questionnaire for that same 
personality trait. When this number is small, predictions are reliable and accurate. 

* SD (Standard deviation): gauges the variation in the mean error of the difference between the 
expected and actual values. When this score is low, there are few discrepancies in predictions for the five 
personality characteristics. 


In order to obtain a more accurate result, we have assumed that unifying the precision of the 
obtained results can provide a more exact metrics’ values since the precision of the values produced by 
the Al-based tool (2 digits after the decimal point) and those returned by the Questionnaire (1 digit after 
the decimal point) are not the same. Hence, the mean error and standard deviation were recalculated 
after unifying the values. 

Table 6 summarizes the results of different evaluation metrics that compares personality prediction 
of Symanto Al based solution to that of the Big Five Questionnaire. 
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Tabl 
Evaluation metrics results of the comparison between Symanto APIs and the Big Five Questionnaire 
| | ME (unified SD (unified 
HPT LPT = = ME ~~ SD i + 
precision) precision) 
Student 1 True True | 0,0320 | 0,0164 0,0200 0,0447 
Student 2 False False | 02180  — 0,1003 0,2000 0,1000 
Student 3 True True | 0,0500  —0,0339 0,0600 0,0548 
Student 4 True True | 0,0460 0,0152 0,0200 0,0447 
Student 5 False False | 03420 — 0,0829 0,3600 0,0894 
Student 6 True True | 0,0620 — 0,0370 0,0600 0,0548 
Student 7 False False 00,2140 ~S sé, 1494 0,1800 0,1483 
Student 8 True True | 0,0360 = 0,0134 0,0000 0,0000 
Student 9 True True —0,0540 ———-0,0351 0,0600 0,0548 
Student 10 True True — 0,0760 ~—0,0351 0,0800 0,0447 
All students 7/10 710 = 0,130 ~=———0,0519 0,1040 0,0636 


The Symanto APIs predict correctly the highest and lowest personality traits in seven students 
from a total of ten students. For all students, the mean difference is roughly 0.11, while the standard 
deviation is about 0.05. The model predicts the Big Five personality characteristics well based on initial 
measurements or unified precision metrics. 

The objective of evaluating the correlation between predictions from the Symanto Al-based model 
and those from the Questionnaire is to figure out the accuracy of Symanto APIs for each personality trait. 
Therefore, instead of comparing the values themselves, the purpose is to compare the variety of the 
predictions. This indicates that if the correlation is close to 1 the values predicted by the Al based tool and 
those of the BF Questionnaire are very dependent. In other words, the change in the personality from 
one student to another occurs in exactly the same way even if the predicted values are not completely 
identical to those identified by the BF Questionnaire. This also means that for each personality trait, there 
is a very strong correlation between the predictions made by the Al-based model and the outcomes of the 
Questionnaire. The correlation results are shown in Table 7. It was calculated both separately for each 
personality trait as well as overall for all personality traits. 


Table 7 
Correlation between Symanto predictions and Questionnaire results 


Correlation Correlation (unified precision) 
Extraversion 0.4731 0.4572 
~ Openness 0,8402 0,8486 
Conscientiousness 0,4691 0,4839 
Neuroticism 0,3573 0,3164 
Agreeableness 0,7114 0,7578 
All traits 0,6143 0,6168 


As shown in the previous table, the correlation values of personality traits in the Symanto model 
range between 0.31 and 0.84. The correlation coefficient for all traits is approximately 0.61 based on the 
initial values and on the unified precision values. In general, we cannot deny the relationship between 
the model's predictions and those calculated by the Questionnaire. However, the correlation is not very 
strong, balanced around 0.6. 

Even the correlation between results indicated by the Questionnaire and those predicted by the Al- 
based model is significant, we cannot consider it substantial. Moreover, we had identified three students 
(students 2, 5 and 7) whose findings were considerably different when comparing Symanto Al-based 
model predictions to the Questionnaire ones (see radar charts in Fig. 3). Consequently, presuming that it 
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contains biased data (see discussion section), we decided to recalculate the correlation after eliminating 
these three students’ data (see Table 8). The correlation increased substantially as a result of this 
modification approaching 0.9. 


Raine between Symanto predictions and Questionnaire results after elimination of biased data 
Correlation Correlation (unified precision) 

Extraversion 0,9591 0,9341 
Openness 0,9787 0,9758 
Conscientiousness 0,9131 0,9303 
Neuroticism 0,8702 0,7727 
Agreeableness 0,8526 0,8775 
All traits 0,9453 0,9309 


We were able to assert the absurdity of the predictions made about the three students by reevaluating 
the data while taking into consideration the samples which show the personality of the student. Table 9 
summarizes the results of this reevaluation while table 10 shows some examples of low and good quality 
samples. 


Table 9 
Number of samples classified by quality after the revision. 
Number of good-quality Number of low-quality 
Student Total by student 
Samples Samples 
Student 1 92 11 103 
Student 2 33 129 162 
Student 3 87 16 103 
Student4 
Student 5 21 87 108 
Student 6 93 10 103 
Student 7 21 57 78 
Student 8 90 8 98 
Student 9 163 23 186 
Student 10 104 15 119 
Total 793 368 1161 
Table 10 
Examples of good and low quality samples after the revision. 
Examples of good-quality Samples Examples of low-quality Samples 
| suggest collaborating with you on this project. The teacher has already shared the correction. 
| must complete the exercise on time. The update is complete. 
| think | can help you debug this code. | will not be present in the next session. 
Discussion 


Concerning the evaluation of Symanto as a personality prediction tool, we can confirm from the 
beginning based on the radar charts representation that Symanto Al-based solution was successful in 
predicting personality traits based on the Big Five Model. Predicted personality of seven students from 
ten was very close to this obtained by the questionnaire. 
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The evaluation metrics defined in this research revealed more important information about the 
evaluated model. For the same seven students, the most and least dominant personality traits were 
correctly predicted. The low values of the Mean Error and the Standard deviation confirm that the 
predictions are very accurate. 

Unifying the precision of the values obtained was supposed to give a more precise comparison 
between the results. However, the findings of the metrics recalculation following the adjustment of values 
with a uniform precision did not significantly affect the initial results. But in any case, we can consider that 
this consequently confirms the initial results. 

The correlation between the two compared results is a very significant metric that calculates 
numerically if the values of personality vary in the same manner in both compared approaches. A more 
important insight that correlation gave us is the variation concerning each personality trait separately. The 
calculation of this metric showed significant correlation based on the results of all students for all personality 
traits. Regarding the assessment of the correlation of each personality trait apart, the correlation was 
raised for the Openness and the Agreeableness, on the other hand it was not very significant for the rest 
of the traits. 

Although overall the results obtained clearly show that the Al-based tool can detect the personality 
of students. We looked for an explanation for the failure of this process for the three students concerned. 
The most likely hypothesis based on how the Al model worked was the quality of the text gathered 
for these students. The text written by a person may in certain cases not reflect its true personality. 
This means that the personality detection algorithm may not be responsible for the error of a detected 
personality from a text that does not contain expressions that really show this personality. The Symanto 
team also recommends using text that specially expresses the person point of view because trough this 
the person shows its personality. According to this, we decided to revise (human revision) all the data to 
check the quality of the text taking in order to classify it according to this criterion. Table 9 represents the 
number of good-quality and low-quality samples for each student after the revision of the data. Examples 
of good and low quality text are shown in table 10. Therefore, based on the results of the new revision 
we decided to exclude the data of the three students and recalculate the same evaluation metrics based 
on the remaining seven students. We considered that the new results are fairer to evaluate the model 
because they are not affected by low quality text. 

The re-evaluation of the model based on the seven students who wrote texts that show more of their 
personality yielded more accurate results. The correlation is really very strong between the personality 
detected by the model and that of the questionnaire. The latter has spread 0.9 for all personality traits while 
it was between 0.85 and 0.97 for separated personality traits. This confirms substantially the accuracy of 
the predictions of the personality. 

Generally, we can assume that this Al-based tool that employs related techniques from NLP to 
identify personality from language expression predicts personality traits exceptionally well. This fact is 
strongly conditioned by having sufficient data collected about each evaluated student, as well as by the 
quality of the data that should include expressions that reveal the student's thoughts and behavior. 

Ultimately, as understanding students’ personalities is very important in the teaching and learning 
context. The present study shows that using automated text-based personality assessment combined 
with good quality text of student collected from social media-based environment can help in detecting 
student personality instead of using the traditional Big Five Questionnaire. and therefore, thanks to these 
tools we can develop specific solutions that automatically detect the learner’s personality in social media- 
based learning environments. 


Conclusions 


In the current study, we collected data from social networks where students discuss about learning. 
We specifically gathered data from Twitter, Google Chat, Instagram, and WhatsApp groups. Data was 
processed using the Al-based Symanto APIs in order to identify students’ personalities. The outcomes of 
this latter were analyzed to determine if the combination of student data from learning environments and 
this tool can help in identifying student’s personality. The results from the Symanto Al-based tool were 
compared to results from the personality test Questionnaire. 

According to the study’s outcomes, Symanto APIs can be really helpful in determining a student's 
personality based on their interactions in learning environments. Most of the student’s personality traits 
were implied in their written text. Moreover, text quality is the most important factor in determining a 
student's personality. 
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The evaluated tool accurately identified the dominant personality characteristic and almost every 
other trait with a very low error value. Furthermore, the correlation between projected personality from text 
and those found by the Questionnaire was very significant (around 0,9) after omitting three students who 
were considered as outlier samples. Either for personality in general or for individual personality traits, the 
correlation value is considerably strong. 

This experiment also shows that not all of the students’ personalities can accurately be identified 
from text. Out of 10 students, three had personalities that the Al-based tool was unable to identify. This 
failure can be explained by the text itself more often than by the tool's inefficiency. The data of these 
students does not contain sufficient expressions that show explicitly students’ personality traits. Therefore, 
it is strongly recommended to check the quality of the text before processing it by the model to detect the 
personality. 

In future work, we will evaluate other Al personality detection tools and test them on other data to 
be collected. We will also study other personality detection techniques that are not text-based to ultimately 
build a multimodal solution that detects personality from several data sources. 
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Abstract: Problem-solving and decision-making skills are essential for individuals across various fields. These skills 
emphasize the importance of preparing a generation capable of solving problems and making informed decisions. Therefore, 
this study aimed to learn the publication trends related to problem-solving and decision-making skills for ESD (Education for 
Sustainable Development) from 2013 to 2022 through Bibliometric analysis. In line with the analysis, a VOSviewer software 
was used to graphically analyze the obtained bibliographic data. A total of 1519 documents were also analytically acquired 
from the Scopus database. The results showed a fluctuating trend in the number of publications, with the Journal of Chemical 
Education and Social Sciences being the highest contributor and the most prevalent field of study at 147 and 689 documents, 
respectively. The United States was also ranked first in the documents emphasizing problem-solving and decision-making skills, 
at 512 documents. Moreover, the University of Toronto was the most prolific affiliation, contributing the most publications at 17 
documents. The representatives from Indonesia were also grouped into two institutions in the global top twenty affiliates, namely 
(1) the Indonesian University of Education and (2) the State University of Malang. In line with the results, 159 study experts 
from Indonesia contributed to the analyzed theme, as the top author originated from the United States having 7 documents. 
The top document excerpts were also published 240 times in the journal Expert Systems with Applications. The trend of the 
study visualization subsequently produced 9 clusters, problem-solving and decision-making skills, human, psychology, clinical 
competencies, education, curriculum, support systems, creativity, and content analysis. These results were helpful to relevant 
experts, regarding the analytical trend in problem-solving and decision-making skills, recommending directions for future analyses. 

Keywords: bibliometric analysis, decision-making skills, ESD, problem-solving, VOSviewer. 


Introduction 


Education for Sustainable Development (ESD) is a new paradigm in the educational field, that 
provides opportunities to the younger generation regarding contribution to sustainable development 
(Tripon, 2014). This paradigm was believed to transform the learning mindsets of students, enabling a 
plan for a better life in the future (Jauhariyah et al., 2021). The ESD curriculum implementation also fosters 
student's competencies, skills, empathy, responsibility, problem-solving, democratic decision-making, 
hope, and respect for humanity as well as nature (Laurie et al., 2016). Furthermore, ESD stimulates 
students to question, analyze, think critically, and make informed decisions (UNESCO, 2012). This 
paradigm employs an active learning model, which emphasizes academic participation and collaboration. 

Based on the aforementioned descriptions, problem-solving and decision-making skills are 
the methodological competencies required for future success (Erol et al., 2016). This indicated that 
collaborative decision-making skills were needed in the industrial era 4.0 (Torres, Pimentel and Matias, 
2023). The methodological competencies were also selected as essential and relevant characteristics 
for physics careers (Nasri, Nasri and Talib, 2020), with decision-making skills closely related to the 
abilities of critical thinking (Molokhina, Pishchik and Fomin, 2021) and communication (Tsai et al., 2020). 
Therefore, schools need to promote decision-making skills through classroom learning. Collaboration 
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among students was also expected to become increasingly important in an environment of complex and 
uncertain decision-making. This is because the interaction among various stakeholders and knowledge 
exchange can enhance the processes of making appropriate decisions (Cai et al., 2023). According to 
a study in various fields, the outcomes obtained were significantly related to decision-making skills in 
learning mathematics (Pertl, Zamarian and Delazer, 2017), physics (Silvianty, Suhandi and Setiawan, 
2019), chemistry (Gao, 2015; Lieber et al., 2022; Wilson and Wilson, 2017), medicine (Delavari et al., 
2022), computers (Carlsson, 2019; Chen and Utama, 2022; Méndez et al., 2020), engineering (Fruchter, 
Katz and Grey, 2018), business (De Villiers, Hankin and Woodside, 2016), and nursing (Chen et al., 
2021; Macduff, Stephen and Taylo, 2016; Saab et al., 2021). Kwok and Hodgson (2017) also stated that 
decisionmaking model consisted of six steps, namely identifying problems, generating, evaluating, and 
selecting alternatives, as well as implementing and assessing decisions. 

Several study trends in decision-making and problem-solving skills have subsequently been 
conducted through bibliometrics. This is because bibliometric studies are significantly related to 
decisionmaking processes in various domains, such as engineering (Cai et al., 2023), universities 
(Vilchez-Roman, Sanguinetti and Mauricio-Salas, 2021), health (Holden, Rosenberg and Barker, 2005), 
linguistic probability (Liao, Mi and Xu, 2020), business (Costa et al., 2017; Pacheco-Velazquez et al., 
2023), laboratory safety (Abdullah and Abd Aziz, 2021), and artificial intelligence (Di Vaio, Hassan and 
Alavoine, 2022). These trends are also relevantly associated with problem-solving in the fields of teaching 
physics (Masitoh et al., 2021) and mathematics education (Chen et al., 2021; Suseelan, Chew and Chin, 
2022). A bibliometric study approach is also commonly implemented to emphasize problem-solving and 
decision-making skills as variables yet to be concurrently investigated by previous experts. This explains 
that both skills are necessary in sustainable development education across various fields of knowledge. 
Therefore, this study aimed to identify the trends in publication growth, countries, subject areas, journals, 
productive authors, university affiliations, highly cited articles, and future experimental opportunities, 
regarding the concepts of problem-solving and decision-making skills. 

The main contribution of the analysis is to provide comprehensive information concerning the trends 
in publication growth, subject areas, top countries, university affiliations, journals, highly cited articles, 
productive authors, contributions of Indonesian experts, and future experimental opportunities in the 
field of problem-solving and decision-making skills. In this case, article authors and journal editors were 
essentially expected to possess awareness of the disciplinary landscape and a thorough understanding 
of the analyzed fields, enabling the identification of future experimental opportunities. Policymakers are 
also capable of recognizing leading countries in the analysis of problem-solving and decision-making 
skills, which were implemented in the instructional design of several dimensional educators. Therefore, 
this study aimed to acquire knowledge about problem-solving and decision-making for ESD, through the 
review and evaluation of the research questions: 

1. How has the trend in annual publications, document sources, subject areas, and countries for 
problem-solving and decision-making skills evolved from 2013 to 2022? 

2. How has the distribution of problem-solving and decision-making skills publications evolved 
based on university affiliation? 

3. Who is the most productive writer in the analysis of global problem-solving and decision-making 
skills? 

4. How has the contribution of Indonesian experts evolved in analyzing problem-solving and 
decision-making skills? 

5. Which paper is most cited and related to problem-solving and decision-making skills? 

6. What are the future study opportunities associated with problem-solving and decision-making 
skills? 


Materials and Methods 


A bibliometric analysis approach was employed to draw conclusions based on a collection of 
bibliographic data, exhibiting the structure, social, and author networks, as well as present analytical 
interests in a specific experimental area (Ha et al., 2020; Karakus, Ersozlu and Clark, 2019; Suprapto et 
al., 2021; Zupic and Cater, 2015). The approach also included the statistical analysis of published articles 
and their citations, to measure relevant impacts (Maditati et al., 2018; Saregar et al., 2022) and analyze 
gaps or emerging topics (Gao et al., 2021). In addition, bibliographical analysis implemented the relevant 
information obtained from online databases, enabling scientific study and a global perspective on related 
interest areas (Secinaro et al., 2020). Figure 1 shows the implementation of the five steps emphasizing 
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bibliometric method. 


; Dat ; Dat : 
Study Design ei Data Analysis Wisu ae ation Interpretation 
Figure 1. Bibliometric methodology 
Study design 


The study design was executed by formulating research questions, as well as selecting keywords 
and databases. This indicated the existence of the six questions investigating the following features: 
i) annual publication trends, document sources, subject areas, and countries,; ii) affiliations,; iii) most 
productive authors,; iv) contributions of Indonesian experts,; v) highly cited papers,; and vi) future 
experimental opportunities related to problem-solving and decision-making skills. The design also proved 
that the implemented search keywords were “Problem-solving decision-making skills”, with the Scopus 
database selected as the bibliometric source. 


Data collection 

A title search strategy (problem AND solving AND decision AND making AND skills) AND 
PUBYEAR > 2012 AND PUBYEAR < 2023 were used by the data collection process from the Scopus 
database. This indicated that 1519 documents for 2013-2022 were acquired from the database, including 
journals, conference proceedings, review articles, paper chapters, and books. The documents were also 
downloaded in Comma-Separated Values (CSV) and Research Information System (RIS) formats, to 
obtain article titles, authors, references, and keywords. 


Data analysis 

Data analysis was initiated by exporting CSV and RIS information from the Scopus database 
to Microsoft Excel and VOSviewer. This analysis was conducted to analyze various trends, including 
publication output characteristics, document sources, country and institutional affiliation distribution, 
subject category dissemination, top authors and citations, as well as publicized tendencies spanning 
2013 to 2022. 


Data visualization 

The visualization of data was carried out using VOSviewer software (Van Eck and Waltman, 2010, 
2012) and Microsoft Excel. By implementing VOSviewer, processed RIS metadata was mapped based on 
network, overlay, and density visualizations. Meanwhile, the visualization, emphasizing Microsoft Excel, 
was used to describe the study trends presented as tables and diagrams. 


Interpretation 

Data visualized by VOSviewer were analyzed and interpreted regarding the number of clusters 
formed, old and recent studies, and density visualization, which specifically exhibited saturation and future 
experimental opportunities. 
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Results 


Annual publication trends between 2013-2022 
The trend of publication productivity over ten years (2013-2022) was based on the Scopus database, 
emphasizing relevant fluctuations. In this case, the analytical trends of problem-solving and decision- 
making skills in various disciplines had a 10-years fluctuation. This was evidenced by the document 
elevations in 2019, 2020, 2021, and 2022 at 196, 172, 197, and 201 publications, respectively. Figure 2 
presents the annual publication trends. 
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Figure 2. Annual publication trends for the period 2013-2022 


Main Source Document 

The article documents were sourced from journals, proceedings, paper chapters, reviews, and 
books. Most of these sources focused on problem-solving and decision-making skills from various fields, 
including education, computer science, engineering, medicine, nursing, public health, psychology, etc. 
Table 1 shows the twenty most prolific sources for publishing articles related to problem-solving and 
decision-making skills. 


Table 1 
Distribution of documents by related sources from 2013-2022 
Sere SJR index Number of 
(Scimagojr 2022) document 
Journal of Chemical Education 0.56 (Q2) 147 
ASEE Annual Conference and Exposition Conference Proceedings 0(-) 40 
Journal of Physics Conference Series 0.18 (-) 18 
Advances In Intelligent Systems and Computing 0(-) 16 
Nurse Education Today 0.95 (Q1) 10 
BMC Medical Education 0.91 (Q1) 10 
Plos One 0.89(Q1) fe) 
Proceedings of the European Conference on Games-Based Learning 0(-) 8 
International Journal of Environmental Research and Public Health 0.83 (Q2) 8 
Sustainability Switzerland 0.37 (Q2) 7 
AIP Conference Proceedings 0.16 (- ) 7 
International Journal of Engineering Education 0.26 (Q2) 6 
Frontiers In Psychology 0.89 (Q2) 6 
ACM International Conference Proceeding Series 0.21 (-) 6 
Journal of Vocational Rehabilitation 0.38 (Q2) 5 
Journal of Veterinary Medical Education 0.45 (Q3) 5 
Journal of General Internal Medicine 1.81 (Q1) 5 
Journal of Engineering Education Transformations 0.21 (Q3) 5 
Journal of Dental Education 0.56 (Q2) 5 
Thinking Skills and Creativity 1.15 (Q1) 4 
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Documents based on Subject Area 

A maximum of 689 problem-solving and decision-making skills publications were obtained for the 
2013-2022 documents emphasizing the subject areas within the field of social sciences. This indicated 
that “Medicine”, “Engineering”, ‘Computer Science”, “Psychology”, “Chemistry”, “Business, Management, 
and Accounting”, “Health Professions”, “Nursing”, and “Arts and Humanities” had 364, 246, 237, 162, 153, 
94, 79, 78, and 74 publications, respectively. Figure 3 illustrates the document classification emphasizing 


subject area. 
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Figure 3. Subject area 


Top Publications by Country 

Based on country classification, a total of 1519 documents were distributed, with 512, 131, 93, 
76, 60, 55, and 42 articles originating from the United States, United Kingdom, Australia, Canada, India, 
Spain, and Indonesia, respectively. This proved that Indonesia, as a country on the Asian continent, 
participated in the study trends of America, Europe, and Africa regarding the analysis of problem-solving 
and decision-making skills. Figure 4 shows the distribution of the top 20 of 102 countries in the publication 
of problem-solving and decision-making skills. 
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Figure 4. Top countries publication on problem-solving and decision making skills 
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University Affiliation 
Figure 5 presents the distribution of the top 20 university affiliations emphasizing problem-solving 
and decision-making skills. 


Number of Publication 


Affiliation 


Figure 5. Top affiliation 


The distribution of publications was observed from various university affiliates, namely; America, 
Malaysia, Indonesia, Iran, Australia, England, Africa, etc. In this case, the University of Toronto was the 
most productive institution with 17 published documents on the Scopus database, accompanied by the 
Universities of Queensland, Melbourne, and Curtin having 13, 12, and 12 articles, respectively. The Tehran 
and Indonesian Universities of Medical Sciences and Education also had 11 documents, accompanied by 
the higher institutions of Pittsburgh, Michigan, and Malang (state) having 10 publications. According to the 
results, Indonesia contributed two institutions to the top 20 affiliates, namely (1) The Indonesian University 
of Education, and (2) The State University of Malang. 

Based on the Scopus database from 2013 to 2022, the top 10 of 39 Indonesian university affiliates 
emphasizing the publication of problem-solving and decision-making skills in learning processes were 
observed. This indicated that the Indonesian University of Education was ranked first, accompanied by 
the Universities of Malang (state), Bina Nusantara, Mataram, Sebeleas Maret, and Semarang (state). 
Table 2 presents the top ten affiliates from Indonesia. 


Table 2 
Top ten affiliations of the University of Indonesia 
No Affiliation Number of documents 
1 Indonesia University of Education 11 
2 State University of Malang 10 
3 Bina Nusantara University 4 
4 University of Mataram 2 
6 Sebelas Maret University 2 
7 State University of Semarang 2 
8 Kanjuruhan University Malang Z 
9 Siliwangi University 1 
10 University of Technology 1 
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Top 10 Authors 

According to the evaluation of the top 10 authors regarding the most publications, Shogren, K.A. 
was ranked first with 7 documents. This was accompanied by Yousefi, M., Suhandi, A., Issa, T., Hooshyar, 
D., and Ahmad, R.B., which had five (5) documents each. York, S., Shultz, G.V., Raley, S.K., and Horng, 
S.J. also possessed four publications each. In line with the results, an Indonesian writer, namely Suhandi, 
A. from the University of Education, analyzed decision-making and problem-solving skills. However, many 
universities in the country did not publish relevant documents based on the Scopus database. Table 3 
illustrates the details for the top 10 authors with the highest number of publications. 


Lope of authors with a minimum of four published documents 

No Author Affiliation eee | ee 
documents citations 

1 Shogren, K.A. University of Kansas, Lawrence, United States 7 107 

2 Yousefi, M. Universiti Tenaga Nasional, Kajang, Malaysia 5 186 

3. Suhandi, A. Indonesia University of Education, Bandung, Indonesia 5 7 

4 Issa, T. Curtin University, Australia 5 2 

5 Hooshyar, D. University of Malaya, Kuala Lumpur, Malaysia 5 186 

6 Ahmad, R.B. University of Malaya, Kuala Lumpur, Malaysia 5 186 

7 York, S. University of Nevada, Las Vegas, United States 4 231 

8 — Shultz, G.V. University of Michigan, Ann Arbor, United States 4 67 

9 Raley, SK. University of Kansas, Lawrence, United States 4 88 

National Taiwan University of Science and Technology, 
10  Horng, S.J. 4 177 


Taiwan 


Contribution of Indonesian Experts Evolved in Problem-solving and Decision-making skills 

Based on the Scopus database spanning from 2013 to 2022, the top ten of the 159 authors that 
published the articles exploring problem-solving and decision-making skills in learning were identified. In 
this case, Suhandi from the Indonesian University of Education mostly published problem-solving and 
decision-making themes. However, the entire learning processes emphasizing the themes remained 
inadequate and limited. Table 4 presents the top 10 authors from Indonesian university affiliates. 


Table 4 
Top fen author contributions from university affiliates in Indonesia 

No Author Affiliation Siem | 

citations 

| Suhandi, A. Indonesia University of Education 5 7 

2 Samsudin, A. Indonesia University of Education 3 3 

3 Syaodih, E. Indonesia University of Education 2 3 

4 Suwono, H. State University of Malang 2 2 

5 Ningsih, A. R. Indonesia University of Education 2 3 

6 Kaniawaiti, |. Indonesia University of Education 2 3 

7 Hakim, A. University of Mataram 2 35 

8 Zarlis, M. North Sumatra University q 0 

i) Zarkasi, A. C University- Nusantara PGRI Kediri 1 0 

10 = Yuliardi, R. STKIP Muhammadiyah Kuningan 1 1 


Document Citation 
According to the Scopus database, the top 10 documents with consecutive highest citations were 
as follows, Liu, Chan and Ran, 2016 (NC=240), Leider and Griffiths, 2019 (NC=220), Ellis et al. 2017 


www.ijcrsee.com 
407 


Ubaidillah, M. et al. (2023). Problem solving and decision-making skills for ESD: A bibliometric analysis, International Journal 
of Cognitive Research in Science, Engineering and Education (IJCRSEE), 11(3), 401-415. 


(NC=212), Dineen-Griffin et al. 2019 (NC=144 


Orgill etal. (2019; NC=107), Hooshyar et al. 2016 (NC=92), 


); 
Kazancoglu and Ozkan-Ozen, 2018 (NC=91), Kegler, Stone and Holland, 2017 (NC=89), Opstoel et al. 
2020 (NC=83), and Devan et al. 2018 (NC=81). Table 5 illustrates the information on the documents with 
the most citations. 


Table 5 
A breakdown of the top 10 documents on Scopus by number of citations 
No Authors Year Title Source al el 
Citation 
Decision-making for the selection of cloud 
: vendor: An improved approach under group Expert Systems 
{ paletat aie decision-making with integrated weights and with Applications id 
objective/subjective attributes 
: Resource-rational analysis: Understanding : 
2 — : 2019 human cognition as the optimal use of limited cea a” 220 
Griffiths : Brain Sciences 
computational resources 
Beyond Risk and Protective Factors: An Perspectives on 
3 Ellis et al. 2017 Adaptation-Based Approach to Resilience Psychological 212 
Science 
Helping patients help themselves: A systematic 
4 Dineen-Griffin 9019 review of self-management support strategies in PLoS ONE 444 
et al. primary health care practice 
Introduction to Systems Thinking for the Journal of 
5 Orgill et al. 2019 Chemistry Education Community Chemical 107 
Education 
Applying an online game-based formative 
Hooshyar et assessment in a flowchart-based intelligent Computers and 
6 2016 j; ; ; 92 
al. tutoring system for improving problem-solving Education 
skills 
Analyzing Workforce 4.0 in the Fourth Industrial Journal of 
Kazancoglu & Revolution and proposing a road map from an __ Enterprise 
7 2018 ; poi : 91 
Ozkan-Ozen operations management perspective with fuzzy Information 
DEMATEL Management 
Trends in suicide by level of Urbanization - Morbidity and 
8 Kegler et al. 2017 ~—_ United States, 1999-2015 Mortality Weekly 89 
Report 
hia aie appeals physical European 
S  |Opstoeletal. || 2090: | SC aes eee enh Physical 83 
Education Review 
What works and does not work in a self- 
40° | Devansten 2018 management intervention for people with Physical Therapy 81 


chronic pain? Qualitative systematic review and 
meta-synthesis 


Mapping Problem Solving and Decision Making with VOSviewer 
Co-occurrence Analysis of Keywords 
The classification of study subjects related to problem-solving and decision-making skills was 


interestingly emphasized, as a minimum number of 58,017 keywords occurred, with 743 words meeting 
the threshold. Network visualization was also categorized into 9 clusters, 42399 links, and 101961 total 
bond strengths. In this case, the red, green, blue, yellow-green, purple, deep sky blue, orange, Mocca, and 
light purple clusters prioritized problem-solving and decision-making skills, humans, psychology, clinical 
competencies, education, curriculum, support systems, creativity, and content analysis through 170, 146, 
127, 126, 85, 47, 36, 5, and 1 items, respectively. Regarding the results, the bigger circle led to the high 
frequency of keywords in the document (Zhang et al., 2022). Figure 6 presents the network visualization 
display using VOSviewer, emphasizing the most repeated keywords and networks. 
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Figure 6. Network visualization of co-occurrences with index keywords. 


Overlay Visualization 

The overlay visualization was symbolized by the colors showing yearly experimental trends, where 
purple and yellow prioritized years of study and recent analysis, respectively. These recent trends focused 
on the following terms, namely collaborative learning, gender, e-learning, employee, academic system, 
social media, formative assessment, project-based education, human experiment, creativity, quality of 
life, child, educational technology, grounded theory, career, theoretical study, software, games, primary 
care, anxiety, and machine knowledge. The overlay visualization of co-occurrence is shown in Figure 7. 
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Figure 7. Overlay of visualization of co-occurrences with index keywords. 
Density Visualization 


Density visualization was responsible for the saturation of a topic, where the blue, yellow, and 
red colors were emphasized yet to be analyzed, rarely studied, and highly evaluated, respectively. This 
indicated that brighter image colors caused the frequent use of the analyzed term in a relevant experiment. 
The distribution of keyword density also illustrated the frequency level of study topics (Huang et al., 
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2020). For example, problem-solving, human, and decision-making were often analytically implemented, 
while the rarely analyzed relevant terms focused on problem-based learning, collaboration, sustainability, 
metacognitive skills, qualitative and quantitative analyses, creativity, life competencies, computational 
thinking, game-based education, human-computer interaction, childhood, virtual patients, expertise, as 
well as analytic and hierarchy processes. Figure 8 presents the density visualization of decision-making 
and problem-solving skills based on keyword index. 
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Figure 8. Density visualization of topic decision-making and problem-solving based on keywords 
index. 


Discussions 


According to Figure 2, a fluctuating growth for the keywords “problem-solving” and “decision- 
making skills”, was observed regarding the publication trend from 2013 to 2022, with the entire productivity 
remaining relatively positive. This indicated significant document elevations from 2017-2019 and 2020- 
2022, underscoring the continuous development of the study topic and providing potential for subsequent 
exploration of novel aspects. Furthermore, Table 1 showed that the Journal of Chemical Education was the 
leading source, publishing 147 articles on “problem-solving” and “decision-making skills”. These articles 
extensively explored the interplay between both skills in handson and mindson laboratory activities. For 
instance, laboratory experiment activities in organic chemistry were commonly problem-based, regarding 
the following performances: (1) developing students information literacy skills (Shultz and Li, 2016); 
(2) guiding inquiry lab practices in the quantitative determination of aluminum within deodorant brands 
(Sedwick et al., 2018); and (3) integrating environmentally friendly chemistry into the curriculum (Aubrecht 
et al., 2019). Problem-solving was also essential for training students in systemic thinking, decision- 
making, and sustainability. 

Based on Figure 3, the publications integrating problem-solving and decision-making skills within 
the “Social Sciences” discipline totaled 689 documents. This indicated that “Social Sciences” focused 
on the widest range of disciplines. It also exhibited the significance of the subject matter prioritizing the 
study, problem-solving, and decision-making skills, which explored the relationship between humans and 
their social interactions. Furthermore, “medicine” generated 364 publications and was closely related to 
problem-solving and decision-making skills, which were integral to effective medical practice. This proved 
that both skills complemented each other and helped doctors in providing high-quality patient care. Medical 
analysis is also presently integrated with artificial intelligence for problem-solving and decision-making 
(Secinaro et al., 2021). From the results, “Engineering” subsequently produced 246 publications, which 
were Closely related to both problem-solving and decision-making skills. In this case, an engineer was 
commonly confronted with the complex tasks requiring technical problem solutions, and rational decision- 
making processes (Kipper et al., 2021). The ‘Computer Science’ discipline was also observed, with 237 
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publications emphasizing problem-solving and decision-making skills. These skills were considered an 
important foundation for developing innovative and effective technical solutions. 

In line with the aforementioned descriptions, “psychology” was another discipline that yielded 162 
publications, with decision-making and problem-solving skills being relevant integral components (Bar-On, 
2010). These skills assisted psychology professionals in handling complex tasks and aiding individuals 
regarding the achievement of mental and emotional well-being. Moreover, the “Chemistry” field produced 
153 documents, with the subject area highly significant as the decision-making and problem-solving skills 
helping scientists and relevant professionals overcome intricate knowledge and management challenges 
(Chen et al., 2021). “Business, Management and Accounting” also generated 94 publications, which were 
pertinent to problem-solving and decision-making skills. These skills emphasized individuals navigating 
complex business challenges, devising effective strategies, and efficiently managing resources to achieve 
organizational goals (Proctor, 2010). Another field, “Health Professions”, also produced 79 documents, with 
the significant subject matter focusing on professional-patients relationships, diagnosing diseases, and 
designing treatment plans, and managing emergency situations. From the results, “Nursing” subsequently 
produced 78 publications, with the subject matter emphasizing the significance of problem-solving and 
decision-making skills. This was because the skills prioritized nurse-patient interaction, problem-solving, 
completing complex tasks, and nursing decisionmaking skills (Anmady and Shahbazi, 2020). In addition, 
“Arts and Humanities” provided 74 publications, whose significant subject matter played an essential role 
in understanding and interpreting relevant works and analyzing cultural contexts. 

According to Figure 4, the United States, among 102 countries, was ranked first in document 
productivity with 512 published articles. This proved that geographic-based information retrieval helped 
to determine the highest publication distribution in a country (Emmer, 2018; Zupic and Cater, 2015). 
Figure 5 also showed that the University of Toronto, as a top university affiliate, was ranked first with 17 
documents. This was accompanied by the contributions of the Universities of Education and Malang, 
which represented the Indonesian institutions possessing 11, and 10 documents respectively. Moreover, 
the contribution of Indonesian experts in the integration of problem-solving and decision-making skills 
remained low according to the Scopus database. The results also indicated the top 10 contributions 
from 39 college affiliations within the country, regarding the publication of problem-solving and decision- 
making skills analyses. From 2013-2022, these publications remained low when perceived as document 
productivity. Therefore, learning should focus on fostering integrative problem-solving and decision-making 
skills for implementing education for sustainable development. This was because the skills were crucial 
in the Industry 4.0 era due to being extensively adopted by workers and managers (Torres, Pimentel 
and Matias, 2023). Learning including real-world problems also stimulated active student engagement 
in applying their individual and cognitive knowledge in collaborative problem-solving (Erol et al., 2016). 

Based on Table 3, Shogren, K.A. from the University of Kansas, Lawrence, United States, was the 
most productive author with a total of 7 documents. The author also had the highest number of document 
citations, totaling 107. Moreover, Yousefi, M., Hooshyar, D., Anmad, R. B., and Horng, S. J. explored 
intelligent guidance by using flowcharts to enhance problem-solving and decision-making (Hooshyar et 
al., 2015). The authors also designed formative assessment with online games in an intelligent tutoring 
system, to enhance problem-solving and decision-making skills (Hooshyar et al., 2016). In line with 
the results, York, S. from the University of Nevada, Las Vegas, United States, published 4 documents 
emphasizing Chemistry Education, with the highest citation of 231. These publications emphasized the 
systematic thinking approaches in Chemistry Education, to strengthen the conceptual understanding, 
problem-solving, and decision-making skills related to global sustainability-based issues (Flynn et al., 
2019; York and Orgill, 2020). The documents also prioritized the systematic thinking approaches in STEM 
Education (York et al., 2019) and learning processes applicable to future chemistry educators, experts, 
and global citizens (Orgill, York and Mackellar, 2019). According to Table 4, the top ten authors from 
Indonesia were Suhandi, A., Samsudin, A., Syaodih, E., Suwono, H., Kaniawati, |., Hakim, A., Zarlis, M., 
Zarkasi, A. C., and Yuliardi, R. This confirmed that Suhandi, A. from the Indonesia University of Education 
was ranked first with 5 published documents, emphasizing physics learning through the implementation 
of systematic thinking, to enhance problem-solving and decision-making skills. Hakim, A. from the 
University of Mataram also published 2 documents in the Journal of Chemical Education, with a total of 35 
citations. This explained that the chemistry education prioritizing students project laboratories enhanced 
understanding, problem-solving, and decision-making skills (Hakim, Kadarohman and Syah, 2016, 2020). 

In line with the aforementioned descriptions, the citations of a document demonstrated its visibility 
and importance as a reference in the academic world. This was because the documents with more 
citations were considered more significant within a specific topic (Zupic and Cater, 2015). The number 
of references obtained by an article also represented recognition, influence, and popularity within the 
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scholarly community. Based on Table 5, the authors focused on problem-solving and decision-making 
skills were exhibited. This indicated that Liu, Chan and Ran (2016) had the highest number of citations 
at 240, exhibiting the following title:’Decision-making for the selection of cloud vendor: An improved 
approach under group decision-making with integrated weights and objective/subjective attributes”. The 
second-ranking publication, authored by Lieder and Griffiths in 2019, also had 220 citations with the 
following title, “Resourcerational analysis: Understanding human cognition as the optimal use of limited 
computational resources”. 

According to Figure 6, a keyword network with a minimum occurrence of five terminologies linked 
to problem-solving and decision-making skills was illustrated. This indicated that the overlay and density 
visualization analyses were used to identify experimental themes, with future reports emphasizing the 
VOSviewer outcomes. Several terminologies such as problem-solving, decision-making, and human, 
were also frequently analyzed. Therefore, the abundance of publications on problem-solving and decision- 
making skills across diverse countries and disciplines prioritized the topic’s significance. This emphasized 
the importance of curricula designation in fostering both skills across various fields. In this case, future 
studies should employ collaborative approaches, gamification, computational thinking, problem-based 
learning, e-learning, creativity, and qualitative designs. 


Conclusions 


In conclusion, this study examined the bibliometric analysis of problem-solving and decision-making 
skills for ESD (Education for Sustainable Development) to provide insights for future analyses. This showed 
that the implementation of both skills was necessary across various educational levels and professions. 
The academic interest in problem-solving and decision-making skills also exhibited fluctuation over the 
previous decade. Based on the results, the Journal of Chemical Education contributed the highest number 
of documents, with social sciences being the most prominent subject area. The United States was also 
the leading contributor of articles, with the University of Toronto being the most productive institution. 
Moreover, the Universities of Education and Malang (state) were the top-contributing affiliations from 
Indonesia. Shogren, K.A., from the University of Kansas, Lawrence, United States, was also considered 
the most productive author with seven documents. In line with the results, Liu, Chan and Ran (2016)was 
the frequently referenced document published in the journal titled “Expert Systems with Applications”. 
According to the recommendations, numerous potential experimental themes were outlined for future 
consideration, specifically within the field of education. These themes underscored curriculum design, 
teaching models, methodologies, assessments, and learning resources, focusing on all levels from 
primary to higher education. The implications of this research can help related researchers find out trends 
in problem-solving and decision-making skills for ESD in the educational sector in general. Education for 
sustainable development should pay special attention to problem-solving and decision-making skills to 
prepare individuals to face future challenges. 
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Abstract: The increasing number of students intending to drop out of universities in Vietnam has raised concerns. While 
previous studies have addressed factors influencing dropout intentions, several aspects still need to be explored, particularly 
in developing countries like Vietnam. This research provides an overview of the factors influencing students’ dropout intention 
in Ho Chi Minh City. The study employs the Partial Least Squares Structural Equation Modeling (PLS-SEM) approach with a 
survey sample of 804 students from universities in Ho Chi Minh City. The research findings reveal that factors such as Lack of 
university commitment (LUC), degree and course commitment (DCC), ineffective time management (ITM), curriculum design 
(CD), Ineffective adaptation to learning environment (IALE), low classroom participation (LCP) and personal circumstances 
(PC) significantly influence students’ dropout intentions. Additionally, factors including skills and attitudes of instructors (SAl), 
instructor support (IS), positive instructor feedback (PIF), university facilities (UF), cultural and social environment (CSE), and 
access to support from academic advisors (ASA) do not show statistically significant relationships with students’ dropout intention. 
Furthermore, the study finds no significant differences in dropout intention based on gender, area, and type of university, except 
for ASA has a differential impact on students’ dropout intentions based on the type of university. The research results provide 
valuable insights for researchers and educational experts to understand better the factors contributing to students’ dropout 
intentions. Moreover, the findings assist educational managers and instructors in developing appropriate support measures 
and interventions to enhance student engagement throughout their academic journey. Finally, the study discusses limitations 
and suggests future research directions. 

Keywords: Dropout intentions, higher education, Ho Chi Minh City. 


Introduction 


Education is one of the foremost concerns for countries worldwide. The sustainable development of 
a nation relies not only on its economic, social, and cultural conditions but also on improving its education 
system. Especially with the advent of the Fourth Industrial Revolution, the role of education Is increasingly 
emphasized in developing a high-quality workforce. There has been an increase in the number of students 
enrolling in higher education institutions annually in foreign countries. However, the number of students 
who want to leave university without obtaining a degree has also significantly increased (Schnettler et al., 
2020). Approximately 15% of university students intend to drop out, which has become a severe issue 
(Sheldon and Epstein, 2004). According to the STEM (Science, Technology, Engineering, Mathematics) 
education approach, the estimated dropout rate of students is around 40-50%. The dropout status of 
students not only negatively affects the students themselves and the university and society as a whole 
(Schnettler et al., 2020). 

In Vietnam, universities have also observed numerous cases of student dropouts. For example, 
Industrial University of Ho Chi Minh City has issued warnings to 2,252 students who voluntarily dropped 
out. University of Transport and Communication has warned 2,135 students regarding their academic 
performance, with 257 students facing expulsion. The Ho Chi Minh City University of Technology and 
Education has removed the names of over 450 students forced to discontinue their studies. The Ho Chi 
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Minh City University of Industry and Trade has over 2,500 students with long-standing tuition fee debts, 
putting them at risk of being banned from taking final exams. The University of Sciences - Vietnam National 
University Ho Chi Minh City has decided to expel 454 students and issue academic warnings to another 
605 individuals. Statistical data demonstrates an increasing trend of student dropouts in Vietnamese 
universities, highlighting the urgent need for measures to mitigate this issue. 

Several studies have focused on the factors influencing students’ intention to drop out before 
completing their university education. Orion, Forosuelo, and Cavalida (2014) found that factors influencing 
students’ dropout intentions include school policies and practices, financial resources, academic 
performance, and teaching programs. Willcoxson (2010) concluded that the factors influencing dropout 
intentions differ among students in the first, second, and third years of university. Farr-Wharton et al. 
(2018) demonstrated the impact of lecturer-student exchange (student-LMX) on engagement, course 
satisfaction, achievement, and intention to leave university among 363 students in an Australian university. 
Schnettler et al. (2020) indicated that costs, age, and difficulties in the learning process tend to make 
students more likely to drop out. Lundquist, Spalding, and Landru (2002) concluded that females are more 
prone to dropping out than males, and factors such as lack of faculty support, unresponsive faculty to 
phone/email inquiries, and complicated faculty-student interactions increase students’ inclination to leave 
the university. Bakker et al. (2021) found that supervisor and co-worker support are negatively related 
to the intention to leave among nursing students. During the Covid-19 pandemic, several studies have 
explored the factors influencing students’ dropout intentions. Chi, Randall, and Hill (2021) showed that 
the COVID-19 pandemic affects students’ mental health and dropout intentions, with those experiencing 
anxiety or depression symptoms and burnout being more likely to consider dropping out compared to 
those without mental health issues. Mtshweni (2021) investigated the factors influencing the intention 
to drop 955 students from a university in South Africa, including social adjustment, personal-emotional 
adjustment, institutional attachment, and socioeconomic status. Baalmann et al. (2022) demonstrated 
that parental educational aspirations, students living in partnerships, and close friends have an impact 
on students’ dropout intentions among a sample of 7,169 students in a German university. Matteau et al. 
(2023) revealed that excessive commitments and conflicts between work, study, and personal life are 
associated with higher levels of psychological stress and the intention to leave university. 

The literature review shows that research on students’ dropout intentions has received significant 
attention from scholars worldwide. The factors influencing students’ dropout intentions are diverse and 
depend on each country’s timeframe and organizational cultural characteristics. Some factors influencing 
dropout intentions mentioned by Willcoxson (2010) are general, comprehensive, specific, and relevant to 
the Vietnamese context. However, Willcoxson (2010) examined the differences in factors affecting dropout 
intentions across semesters and among first-, second-, and third-year students but needed to determine 
the impact level of each factor on students’ dropout intentions. Moreover, the factors mentioned, such as 
commitment to the institution, degree/course commitment, time management, teaching skills and attitudes 
of instructors, accessibility and support from instructors, course design, feedback, ineffective adaptation 
to the learning environment, class participation, infrastructure, socio-cultural environment, accessibility 
and support from counseling, and personal circumstances that align with the context and culture of first, 
second, and third-year students in Vietnamese universities. 

This research aims to provide an overview of the factors influencing students’ dropout intentions 
in universities within Ho Chi Minh City. While many studies have identified a range of factors that may 
contribute to students’ dropout intentions, there still needs to be clear validation regarding the level of 
impact of each factor. Therefore, the contribution of this study is to clarify the degree of influence of 
these factors on students’ dropout intentions in universities within Ho Chi Minh City, where extensive 
validation studies still need to be completed. This research utilizes a non-probability and convenient 
sampling method to collect data from the survey participants easily. The study’s geographical scope is 
limited to the inner city and suburban areas of Ho Chi Minh City. The research has two main objectives: 
1) identifying the factors influencing students’ dropout intentions within the Ho Chi Minh City area, and 2) 
proposing managerial implications to improve these factors to reduce students’ dropout intentions within 
the Ho Chi Minh City area. 


Definition of dropout intentions 

According to, Pijl, Frostad, and Mjaavatn (2014), early dropout refers to needing to complete an 
educational program or complete it with significant delays. Additionally, Schwab (2018) suggests that 
when individuals intend to leave school, they quickly focus on the desire to discontinue their education. 
Therefore, dropout is considered the final step in intending to leave school before early dropout occurs. 
According to, Gury (2011), dropout occurs when students discontinue their studies without intending to 
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continue in the initially registered field of study or the institution they attend. Fitzpatrick and Yoels (1992), 
define dropout as students who leave an educational institution without completing their program within 
the next four years, regardless of whether they return to school later and graduate. Furthermore, dropout 
can refer to individuals participating in a school course who do not wish to complete the high school 
program within five years (Pijl, Frostad, and Mjaavatn (2014). 

Based on the definitions provided above, the dropout intentions can be defined as not completing 
an educational program or completing it with significant delays, not continuing in the initially registered 
field of study or institution, leaving school without graduating within a specific timeframe, or not wanting to 
complete the academic program within a certain period. 


Research hypothesis development 

Students have various reasons for choosing to attend a university, including personal purposes 
such as academic pursuit, proximity to their residence, reputation, quality of education, and the quality of 
facilities at the institution. Based on their criteria, students can evaluate suitable universities and select a 
school that meets their conditions. The choice of university has a specific impact on students’ subsequent 
intention to drop out, especially in cases where students do not gain admission to their desired university 
and must attend an alternative institution. Willcoxson (2010) also indicates that students are more 
likely to leave university when they lack organizational commitment and receive insufficient guidance 
regarding enrollment choices. This situation commonly occurs among first-year students. From the 
research findings, Willcoxson (2010) determines that when students fail to gain admission to their desired 
university, the likelihood of them forming an intention to drop out increases significantly. Furthermore, 
during their studies at the alternative university, students still hope to gain admission to their initial desired 
university and attend the substitute university as a stepping stone to transfer to another university, thereby 
increasing their intention to drop out (Willcoxson, 2010). Bean (1980) analyzed a model contributing to 
student dropout and found a correlation between student commitment and the intention to drop out. This 
study demonstrates that students need more commitment to the institution to increase their intention to 
drop out and continue their educational journey. 


H1: The lack of university commitment positively impacts student’s dropout intentions in Ho Chi 
Minh City. 


Mackie (2001) demonstrates that students who engage in courses over multiple years face similar 
difficulties as those who have dropped out before completing their studies. However, the remaining people 
exhibit more substantial commitment and attachment to the institution. Students who stay in school are 
more likely to overcome challenges than those who have dropped out (Nieudwoudt and Pedler, 2021). 
Students with explicit purposes for pursuing a specific field of study are more likely to intend to enroll in that 
particular academic program at the university. Yorke and Longden (2008) indicate that strong academic 
commitment is associated with stability and persistence in students’ studies, while weak commitment may 
lead to an intention to drop out. Tinto (2012) also suggests that solid academic commitment positively 
impacts students’ continued engagement in learning activities and reduces the likelihood of dropping out. 
Therefore, universities with a clear commitment to degree programs and the career-related benefits they 
Offer, aligning with students’ prospects, enhance students’ commitment and attachment to the university. 


H2: Degree and course commitment negatively impact student's dropout intentions in Ho Chi Minh 
City 


Swick (1987) argues that many students perceive the academic process as highly stressful. Time 
management is a university counseling service (Macan and Shahani, 1990). Students also need help 
to allocate their time effectively and balance it with work and personal life (Burke et al., 2017). Time is 
a significant factor influencing students’ daily lives. When time management skills are weak, such as 
inadequate time allocation or last-minute cramming for exams, it is discussed as a cause of stress and a 
decline in academic performance (Longman and Atkinson, 1988). The issues mentioned above occurring 
consistently over an extended period can discourage students, resulting in a gradual formation of the 
intention to drop out (Nieudwoudt and Pedler, 2021). Students struggling to balance their personal time 
and study time at the university and those struggling with effective time management are more likely to 
have an increased intention to drop out (Willcoxson, 2010). 
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H3: Ineffective time management positively impacts as students’ dropout intentions in Ho Chi Minh 
City. 


Willcoxson (2010) found that reducing students’ intention to drop out is related to building trust, 
fostering learning expectations, improving teaching quality, providing support, and creating a vibrant 
learning environment and social activities. Additionally, the emerging tendency of students to consider 
dropping out is also linked to the teaching skills and attitudes of the faculty. Students perceive enthusiastic 
support from instructors, a sense of closeness, provision of comprehensive learning materials, and timely 
and positive feedback as factors that reduce their intention to drop out (Willcoxson, 2010). Social support, 
learning experiences, and an engaging learning environment, along with institutional support, are factors 
that influence students’ decision to remain in school (Nieudwoudt and Pedler, 2021). Poor interaction or 
communication with instructors and mentors can lead to students’ intention to drop out (Nieudwoudt and 
Pedler, 2021). 


H4: Skills and attitudes of instructors negatively impact student’s dropout intentions in Ho Chi Minh 
City. 


Glogowska, Young, and Lockyer (2007) demonstrate that students’ determination, career 
commitment, social support, and student services provided by the university contribute to student retention. 
Natoli, Jackling, and Siddique (2015) conclude that student support services offered by the university 
are an essential factor in influencing students’ intention to stay in school. The institution’s provision of 
facilities, faculty, programs of study, student support services, and engagement in academic activities all 
contribute to student retention (Kuh et al., 2007). Instructor members who are willing to address students’ 
concerns and understand their difficulties in learning create enthusiasm for studying and reduce students’ 
thoughts of dropping out. 


Ho: Instructor support negatively impacts student’s dropout intentions in Ho Chi Minh City. 


Willcoxson (2010) argues that carefully designed and logically structured courses with reliable 
information yield high educational effectiveness. Instructors who incorporate real-life examples in their 
lectures help students quickly understand and apply the subject to practical work situations (Willcoxson, 
2010). Furthermore, university support for students to engage in experiential learning and work 
opportunities in companies enhances their knowledge and skills, reducing their intention to drop out. 
A flexible curriculum can make students feel more comfortable dropping out. Rovai and Jordan (2004) 
have demonstrated that program flexibility can increase students’ commitment and intention to continue 
their studies. Bransford (2000) emphasizes the importance of applying knowledge to real-life situations, 
connecting knowledge with reality, and applying it in daily life to help students recognize the value of 
learning and enhance their commitment to education. 


H6: The curriculum design negatively impacts students’ intention to drop out in Ho Chi Minh City. 


Case (2007) demonstrates that feedback is crucial in promoting student improvement by addressing 
errors, lessons’ shortcomings, and areas needing improvement. Faculty support has a positive impact 
on academic performance and student engagement. If instructors fail to meet students’ expectations or 
requirements, harmful or ineffective feedback can lead to disappointment and strengthen the intention to 
drop out (Hausmann, Schofield and Woods, 2007). 


H7: Positive instructor feedback negatively impacts students’ dropout intentions in Ho Chi Minh 
City. 


When participating in courses at school, students may encounter difficulties in comprehending 
knowledge, struggle to adapt and keep up with the teaching methods of instructors, find it challenging 
to understand specialized materials and feel overwhelmed by the workload. These factors can lead to 
student frustration, a lack of self-belief in their ability to perform well in the courses, decreased motivation 
to study, and an increased intention to drop out (Willcoxson, 2010). Eccles and Wigfield (2002) suggest that 
students’ positive adaptation to the learning environment often leads to a more substantial commitment 
to the learning process and a higher likelihood of sustaining their studies and completing the courses. 
Effective adaptation can help students reduce stress and pressure in the learning process, which can 
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contribute to the intention to drop out (Eisenberg et al., 2007). Kember, Biggs and Leung (2004) have 
demonstrated the relationship between adaptation to the learning environment and students’ academic 
performance, showing that well-adapted students tend to have higher grades, more stable academic 
performance, and maintain their intention to study throughout the program. 


H8: Ineffective adaptation to the learning environment positively impacts students’ dropout intentions 
in Ho Chi Minh City. 


According to empirical research surveys, many students who drop out initially show commitment 
but fail to follow through (MacKie, 2001). Regular class truancy and non-participation positively correlate 
to dropout (Willcoxson, 2010). Classroom engagement often provides valuable learning opportunities. 
When students participate or participate minimally, they may take advantage of opportunities to 
understand and acquire the necessary knowledge to achieve better results in assessments (Pascarella 
and Terenzini, 1991). Kuh et al. (2008) have shown that classroom engagement is often correlated with 
academic performance, with lower levels of engagement resulting in poorer academic outcomes and 
higher intentions to drop out. Classroom engagement reflects the commitment to the learning process. 
Students who participate less in class may need more commitment and determination to complete the 
course (Feldman, 1994). 


H9: Low classroom participation positively impacts students’ dropout intentions in Ho Chi Minh City. 


Good facilities create a conducive and comfortable learning environment that caters to the needs of 
students and minimizes the likelihood of students intending to drop out. Good facilities influence student 
satisfaction, impact student confidence (Omar et al., 2009), and shape future planning intentions (Clemes, 
Gan and Kao, 2008). Reynolds (2007) analyzed the correlation between facilities and student recruitment 
and retention. Classrooms that provide a high-quality learning environment, spacious and well-ventilated 
libraries with diverse resources to support learning, and an information technology system that meets 
students’ usage needs have a reverse correlation with students’ intention to drop out (Willcoxson, 2010). 


H10: The university facilities negatively impact students’ dropout intentions in Ho Chi Minh City. 


The cultural and social environment significantly influences students’ dropping out. Students may 
feel helpless, isolated, and unwilling to continue their education when this environment is not friendly. 
Conversely, when positive relationships characterize the environment, students will receive support and 
encouragement to continue their studies. Research has shown that the cultural and social environment 
impacts student satisfaction (Kahu, 2013). According to Willcoxson (2010), minimizing student dropout 
requires providing facilities that meet social needs and are compatible with students’ religious/cultural 
requirements. 


H11: The cultural and social environment negatively impacts students’ intention to drop out in Ho 
Chi Minh City. 


Access to information and support from academic advisors increases student retention (Crosling, 
Thomas and Heagney, 2009). Student retention depends not only on individual factors such as motivation 
and academic achievement but also on external factors such as access to support and resources (Cabrera 
et al., 2006). Access to information, guidance, and counseling from academic advisors and classmates, as 
well as academic and social support services, can be crucial in the decision to continue or withdraw from 
university (Tinto and Pusser, 2006). Students with access to high-quality support services are more likely 
to be motivated and have higher retention rates (Tinto and Pusser, 2006). Students receiving good advice 
from advisors regarding career choices or quickly receiving assistance when needed have a reverse 
correlation with their intention to drop out (Willcoxson, 2010). 


H12: Access to support from academic advisors negatively impact students’ dropout intentions in 
Ho Chi Minh City. 


Students who face financial difficulties often spend more time working than studying (Peltz et al., 
2021). Financial difficulties negatively impact students’ commitment to their studies (Willcoxson, 2010). 
Studies have found that students with high intentions to drop out often face financial hardships and work 
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an average of more than 16 hours per week (Leveson, McNeil and Joiner, 2013). Bean (1980) examined 
the impact of personal circumstances on students’ decision to drop out. The results showed that the 
impact of difficult personal circumstances can lead to an intention to drop out. Pascarella and Terenzini 
(1991) demonstrated that the university environment and students’ circumstances influence the decision 
to continue their education. The results also indicated that personal circumstances can be essential to 
the dropout decision process. Students’ concerns about mental health, physical health, homesickness, or 
accumulating debt positively correlate with their intention to drop out (Willcoxson, 2010). 


H13: Personal circumstances positively impact students’ dropout intentions in Ho Chi Minh City. 


Skills and attitudes of 
management instructors 


2 H3(+) ie 
Personal circumstances H4(-) Instructor support 
\ 


H5(-) 


Lack of university Degree and course Ineffective time 


commitment commitment 


Student’s dropout 
intention 


Access to support from H12(-) 
academic advisors 


H11(-) 


Bea le Low classroom 
University facilities 
engagement 


Figure 1. Proposed research model 


Positive instructor 
feedback 


Cultural and social 
environment 


Materials and Methods 


Process research: A mixed-methods research approach combining qualitative and quantitative 
research methods was used in this study. 

Preliminary qualitative and quantitative research: A group interview method was employed with 
15 students in the qualitative phase of the study. The research topic involved collecting opinions from first-, 
second-, and third-year students at public and private universities in suburban and urban areas of Ho Chi 
Minh City. The group discussion aimed to identify factors influencing the intention to drop out and refine 
the measurement scales of the research concepts to align with the research context. The results of the 
interviews were synthesized and adjusted to form a draft measurement scale to support the preliminary 
quantitative research and the formal quantitative research. Subsequently, a survey was conducted with 
80 students to evaluate the reliability using Cronbach’s Alpha coefficient and perform Exploratory Factor 
Analysis (EFA) to examine the convergent and discriminant validity of the measurement scale. 

Formal quantitative research: The study utilized the Bootstrapping technique with a sample size 
of N = 5000 to test the hypotheses. This step was employed to evaluate the measurement model and the 
structural model: 

The measurement model was assessed by examining measurement scale reliability, composite 
reliability, convergent validity, and discriminant validity. To ensure the reliability of the measurement scales, 
Cronbach's alpha coefficient and Composite Reliability (CR) should exceed 0.6 (Hair Jr et al., 2009). The 
Average Variance Extracted (AVE) of each construct in the model should be greater than 0.5, based on 
the criteria proposed by Shiau, Sarstedt and Hair (2019). The study followed the criteria of Fornell and 
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Larcker (1981) to test the discriminant validity of the measurement scales, where the square root of the 
AVE of each construct should be greater than the correlation coefficient between that construct and the 
other constructs in the model. 

The structural model was evaluated based on criteria such as the coefficient of determination (R2), 
predictive relevance (Q2), and effect size (f*). The coefficient of determination (R2) values of the model 
were interpreted as follows: weak (R2 = 0.02), moderate (R2 = 0.16), and robust (R2 = 0.26) explanations 
of the model variance (Cohen, 2013). The Stone-Geisser Q? criterion was used for predictive relevance 
assessment, following the evaluation standards proposed by Henseler, Ringle, and Sinkovics (2009): 
weak prediction (Q2 < 0,02); moderate prediction (Q2 within [0,02; 0,35]), and strong prediction (Q2 > 
0,35). Lastly, the effect size (f?) between corresponding components was examined, with weak effect (f 
= 0.02), moderate effect (f? = 0.15), and substantial effect (f? = 0.35) based on the criteria of Henseler, 
Ringle and Sinkovics (2009). 

Scale measurement: The research model consists of 13 research constructs. The dependent 
variable is Dropout Intentions, which was adopted by Farr-Wharton et al. (2018). The independent 
variables include Lack of university commitment, Degree and course commitment, Ineffective time 
management, teaching skills and attitudes of instructors, Instructor support, Curriculum design, Positive 
instructor feedback, Ineffective adaptation to the Learning Environment, Low Classroom Engagement, 
University facilities, Cultural and social environment, Access to support from academic advisors, and 
Personal Circumstances. These independent variables were inherited and adjusted from the study by 
Willcoxson (2010). These independent variables were inherited and adjusted from the study by Willcoxson 
(2010). There are a total of 74 observed variables, and they were measured using a 5-point Likert scale: 
(1) Strongly Disagree, (2) Disagree, (3) Neutral, (4) Agree, and (5) Strongly Agree (see Table 1). 


Table 1 
Scale measurement 
Constructs Symbol No. . Scale sources 
observations 
Student’s dropout intentions SDI 4 Farr-Wharton et al. (2018) 
Lack of university commitment LUC 5 
Degree and course commitment DCC 3 
Ineffective tine management ITM 4 
Skills and attitudes of instructors SAI 7 
Instructor support IS 5 
Curriculum design CD 3 
COMME ater! aimed 3 oe 2 Willcoxson (2010) 
Ineffective adaptation to learning IALE 7 
environment es ee BC 
Low classroom participatio LCP 10 
University facilities UF ae: 
Cultural and social environment CSE 7 
Access to support from academic ASA 7 
advisors 
Personal circumstances PC 5 


(Source: own author) 


Formal sample 

Survey Sample Criteria: First- year, second- year, and third- year university students studying at 
expected public and private universities located within the inner city and suburban areas of Ho Chi Minh 
City. This study did not survey fourth-year students as they rarely intend to drop out. 

Sampling Method: The study employed a non-probability convenience sampling method. The 
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survey questionnaire was distributed directly and online through Google Forms at various universities in 
Ho Chi Minh City. The survey was conducted from February 13, 2023, to March 16, 2023. 
Data Analysis Method: The study utilized Partial Least Squares Structural Equation Modeling (PLS- 
SEM) to analyze the data. This method was chosen due to its advantage in handling small sample sizes 
and data that do not follow normal distribution assumptions (Shiau, Sarstedt and Hai, 2019). 
Formal Sample: The survey results yielded 804 valid responses. Therefore, the study used 804 
observations as the formal sample for this research. 


Results 


Sample characteristics 

Gender breakdown with the number of female students being 392 (48.8%) and the number of 
male students being 412 (51.2%). Next is the breakdown of students by academic year, with 239 (29.7%) 
first-year students, 261 (32.5%) second-year students, 304 (37.8%) third-year students, and no fourth- 
year students. Following that is the breakdown of students by major, with the corresponding number 
of students in each major. The majors listed are Engineering with 163 students (20.3%), Economics 
- International Trade with 133 students (16.5%), Business - Management with 158 students (19.7%), 
Foreign Languages with 63 students (7.8%), Information Technology (IT) with 64 students (8.0%), Social 
Sciences and Humanities with 25 students (3.1%), and other majors with 198 students (24.6%). Next is 
the geographical breakdown, with the number of suburban students being 284 (35.3%) and the number 
of downtown students being 520 (64.7%). 

The university group includes various universities, with the corresponding number of students in 
each university. The listed universities are Open University of Ho Chi Minh City with 70 students (8.7%), 
Ho Chi Minh City University of Transport with 60 students (7.5%), Industrial University of Ho Chi Minh City 
with 184 students (22.9%), Ho Chi Minh City University of Technical Education with 71 students (8.8%), 
Nong Lam University with 80 students (10.0%), Van Lang University with 71 students (8.8%), UEF School 
of Economics and Finance with 53 students (6.6%), HUTECH University with 62 students (7.7%), Nguyen 
Tat Thanh University with 79 students (9.8%), and FPT University with 74 students (9.2%). Finally, the 
group with intentions to drop out is divided into two categories: those with intentions to drop out, totaling 
206 (25.6%), and those without intentions to drop out, totaling 598 (74.4%). 


Table 2 
Participants’ Characteristics 
Characteristics Frequency (%) 
Cae Femde 892 48.8% 
Male 412 51.2% 
First-year 239 29.7% 
Student Second- year 261 32.5% 
Third- year 304 37.8% 
Fourth- year 0 0.0% 
Engineering 163 20.3% 
Economics - International Trade 133 16.5% 
Business - Management 158 19.7% 
Field of study Foreign Languages 63 7.8% 
Information Technology (IT) 64 8.0% 
Social Sciences and Humanities 25 3.1% 
Other a : _s 198 24.6% 
hie Suburban 284 35.3% 
Urban 520 64.7% 
HCMC Open University 70 8.7% 
University of Transport and Communication 60 75% 
Industrial University of HCMC 8 22.9% 
HCMC University of Technology and Education 1 8.8% 
Cinivaeel nests Nong Lam University 80 10.0% 
Van Lang University 1 8.8% 
University of Economic and Finance 53 6.6% 
HCMC University of Technology 62 11% 
Nguyen Tat Thanh University 79 9.8% 
FPT University 14 9.2% 
Have you had No 598 14.4% 
intentions to drop out? Yes 206 25.6% 


(Source: own author) 
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Scale evaluation: In the PLS-SEM method, the outer loadings criterion is used to evaluate the 
importance of predictor variables in the model. According to Henseler, Ringle and Sinkovics (2009), factor 
loadings > 0.5 are considered. Factor loadings below 0.5 will be excluded from the measurement scale 
in the model. 


Table 3 
Scale reliability 
: P Average variance 
Constructs No. observations | Cronbach's alpha extracted (AVE) _ 
DCC 3 | 0.797 0.712 
UF 4 0.873 0.722 
LGP 10 0.665 0.748 
IS ) 0.684 0.7 
IALE 7 0.685 0.605 
PIF 2 0.85 0.862 
LUC 5 0.654 0.591 
SAI 7 0.881 0.681 
ITM 4 0.83 0.658 
CD 3 0.81 0.725 
ASA 7 0.919 0.749 
PG 5 0.808 0.567 
SDI 4 0.805 0.631 


(Source: own author) 


Table 3 presents the reliability testing results, including Cronbach's Alpha, Composite Reliability 
(CR), and Average Variance Extracted (AVE) for the measurement scales in the model. The statistical 
table shows that the Cronbach's alpha values of the measurement scales are all above 0.7, ensuring 
reliability for use (Nunnally, 1978). Therefore, the variables will be retained and utilized in the subsequent 
steps. Hair et al. (2019), state that a Composite Reliability (CR) value greater than 0.7 ensures reliability. 
Based on the results in Table 3, all measurement scales have CR values above 0.6, except for the LCP, 
IS, ALE and LUC scales. Lastly, the Average Variance Extracted (AVE) for all measurement scales is 
more significant than 0.5, ensuring reliability (Hair et al., 2019). Hence, most measurement scales in the 
research model demonstrate satisfactory reliability. 


Table 4 
err Statistical value 


| Mean | S| __—Factor loadings 


oe dropout inten- 
tions (SDI) 
SDI1: | often think about 


SDI2: | am actively seeking 

job opportunities and alter- 

native learning options, so | 2.044 1.123 0.815 
may leave the university. 


SDI3: There is a Smee f cam 
that | will drop out of univer- 2 pa 1 ptm 0. poy 
sity within the next year. 


SDI4: | am looking for suit- 

able timing to drop out of 1.769 1.092 0.857 
school. 

Lack of university commit- 

ment (LUC) 
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LUC1: | am studying at this 
university as a steppingstone 
to transfer to another univer- 
sity. 


LUC3: | am attending this 
university because | did not 
meet the requirements of 
other preferences. 


Degree and course com- 
mitment (DCC) 


DCC1: | have obvious rea- 
sons for studying at this uni- 
versity. 


DCC2: | can enroll in the 
course/program that | have 
chosen. 


DCC3: | know what profes- 
sion | want to pursue in the 
future. 


Ineffective time manage- 
ment (ITM 


ITM1: It is difficult to balance 
personal time and study time 
at the university. 


ITM2: | struggle with manag- 
ing study time effectively. 


ITM3: It is challenging to bal- 
ance family responsibilities 
and university studies. 


ITM4: It is challenging to 
balance work and university 
studies. 


Skills and attitudes of in- 
structors (SAI) 


SAI‘: The professors are en- 
thusiastic and dedicated in 
their teaching. 


SAI2: The professors are 
skilled at explaining things. 


SAI3: The professors always 
strive to make the classes in- 
teresting. 


SAI4: The faculty team clear- 
ly communicates their ex- 
pectations from the students 
right from the beginning. 


SAI5: The professors always 
create a sense of closeness 
with the students. 


Instructor support (is) [| SSSSSC~dSSSSSSSSSS 
IS4: The faculty team is al- 

ways available when | need 3.68 1.699 

them. 

IS5: My professors genuinely 

make an effort to understand 

the difficulties students face aon on 

in their learning process. 


Curriculum design (CO) | | CP 
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CD1: My professors incorpo- 
rate real-life examples into 
their teaching curriculum. 


CD2: What | am learning at 
the university has been re- 
searched and proven. 


CD3: | am satisfied with the 
job experiential opportunities 
introduced by the university. 


Positive instructor feed- 
back (PIF) 


PIF 1: | received helpful feed- 
back on the assessment 
tasks. 


PIF2: | received prompt feed- 
back on the tasks. 


Ineffective adaptation 
to learning environment 
(IALE) 


IALE2: My study program is 
too demanding. 


IALE4: | find it difficult to un- 
derstand various study mate- 
rials. 


IALES: | struggle to adapt to 
the teaching methods at the 
university. 


Low classroom participation 
(LCP 


LCP7: | frequently skip class- 
es. 


LCP8: | don't attend classes 
because the study materials 
are available on the website. 


University facilities (UF) 


UF1: The classrooms pro- 
vide a high-quality learning 
environment. 


UF2: The library is spacious, 
well-ventilated, and offers a 
diverse range of study ma- 
terials. 


UF3: The information tech- 
nology system meets my us- 
age needs. 


UF4: The classrooms are 
very spacious. 

Cultural and social envi- 
ronment (CSE) 


CSE1: The facilities of the 


university meet my social 
needs. 


CSE2: The facilities of the 
university are suitable for my 
religious/cultural needs. 
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ASA1: | receive good advice 

from the university regarding 3.397 0.995 0.893 
career choices. 

ASA2: | receive good advice 

from a career counselor in 

choosing a profession for a08F 1 0.907 
myself. 

ASA3: | receive good advice 

from the university regarding 3.429 1.03 0.92 
career choices. 

ASA4: | easily receive assis- 

tance when needed from the 3.435 0.999 0.842 
management team. 

ASAS: The management 

team is always ready to as- 3.427 1.019 0.754 
sist when | need them. 

Personal circumstances 

(PC) 


PC1: | worry about my men- 
PC2: | worry about my physi- 
cal health. 3.039 1.202 0.821 


PC3: | often feel homesick. 3.213 1.309 0.584 


PC4: | worry about the accu- 

mulating debt while studying 2.976 1.393 0.774 
at university. 

PC5: | have financial issues. | 806s 1.351 0.742 


(Source: own author) 


Table 4 presents the descriptive statistics, standard deviations, and factor loadings of the variables 
after variable elimination. The results show that all factor loadings are greater than 0.7, except for SAI5 
and PC3, but they are retained to ensure content validity. The measurement scales used in the research 
model exhibit convergence. 


Table 5 

The discriminant validity testing 

DccC = PC UF LCP IS ALE PIF LUC. SAl_ ITM. CD ASA. CSE SDI 
DCC 0.844 


PC -0.053 0.753 

UF 0.348 —--0.017 0.85 

LCP. -0.101 = =—0.205 = -0.036 0.865 

IS 0.284 0.013 0.454 0.001 0.837 

IALE 0.017 0.309 0.007 =: 0.23 0.12 0.778 

PIF 0349 0024 0519 0.034 0463 0.147 0.928 

LUC __ -0.047 0.193 -0.066 0.214 -0.022 0.208 0.025 0.769 

SAI 0.396 _—-0.034 0.508 -0.107 0615 0.026 0512 -0.109 0.825 

ITM -0.007 | 0.3 -0.007 021 0.036 0.331 0.08 0.232 -0.01 0.811 

CD 0.424 ~~ -0.022 0.533 -0.088 0.53 0.081 0.622 -0.027 0.669 0.000 0.852 

ASA 0.377 (0.051 0.542 0.059 0472 0.102 0574 0.023 0.494 0.038 0.55 (0.865 

CSE 0.352 —_—0.046 0.669  -0.007 0.429 0.038 0481 -0.016 0.465 0.061 0.491 0.524 0.945 

SDI -0.193 0.262, -0.144 = 0.284 -0.113 0.27  -0.083 0.389 -0.215 0326 -0.19 -0.087 -0.09 0.794 
(Source: own author) 


Table 5 presents the discriminant validity test results for the model’s latent variables using the 
criteria set by Fornell and Larcker (1981). The table shows that all square root of the average variance 
extracted (AVE) values for each research variable are more significant than the correlation coefficients 
between that variable and the remaining variables in the model. Therefore, the measurement scales for 
the research variables all demonstrate discriminant validity. 
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Model evaluation: The estimation results of the model using the Bootstrapping method with a 
sample size of 5,000 are depicted in Figure 2. 


ASA\ 
0.003 (0.952'9 019 (0.652) 


0.268 (0.000) 
0.099 (0.006) 


0.089 (0.008) 


0.172 (0.000) 
~~ 0.068 (0.158) 0.009 (0.833) 
0.012 (0.759) 0.121 (0.001) 
0.114 (0.001) 


-0.089 (0.079) 9.001 (0.976) 


Figure 2. PLS-SEM estimation results 


Table 6 
Hypothesis test results 


, | Estimation P ‘ 
Hypotheses Path relations Beta (8) | Bootstrapping (B) | SD T values Conclusion 
Hr LUC SDI 0.268 0.268 0.038 7.043 0.000 Accepted 
Ho DCC -> SDI —— -0.099 -0.1 0.036 2.724 0.006 Accepted 
H3 ITM -> SDI 0.172 0.172 0.033 5.146 0.000 Accepted 
He  SAI>SDI 0.068 0.064 0.048 1.41 0.158 Rejected 
Hs IS -> SDI 0.012 -0.014 0.038 0.307 0.759 Rejected 
Hg CD -> SDI _ -0.089 -0.088 0.051 ~—- 1.759 0.079 Accepted 
Hr PIF> SDI 0.001_— 0.000 0.020.030.9768 Rejected 
He  -IALE->SDI 0.114, 0116 0.085 3.21 0.001 Accepted — 
Hg LCP-> SDI = 0.121 0.123 0.035 3.43 0.001 Accepted 
Hi0 _UF-> SDI 0.009 -0.012 0.044 0.211 0.833 Rejected 
Hi CSE>SDI 0.008 0.008 0.042 0.06 = 0.952 ~—sRejected 
Hi2 _ASA-> SDI 0.019 0.022 0.041 0.451 0.652 Rejected 
H43 _ PC-> SDI 0.089 0.09 0.033 2.663 0.008 Accepted 
RO 
P f@ac->spi = 0.01; fec->spi = 0.009; faicp->spi = 0.018; fiaLe->spi = 0.015; f.uc>spi = 0.09; fasai-spi = 
| 0.003; féit->sp1 = 0.034; f2cp->sp1 = 0.005 
Stone- 
Geisser’s Q? 0.178 


(Source: own author) 


The quality of the proposed model is assessed through the R2 values and the Stone-Geisser index 
(Q2). Table 6 shows that the R2 value for SDI is 0.30, more significant than 0.26. According to Cohen 
(2013) evaluation criteria, the model’s predictive power is considered strong. The Stone-Geisser value 
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from Q?SDI is 0.178, falling within the range of (0.02-0.35). Following the evaluation criteria of Henseler 
and Chin (2010), the model's predictive ability is considered moderate. Additionally, the effect size (f?) of 
the factors influencing students’ intention to drop out is evaluated as weak. According to, Hair et al. (2019), 
the influence of factors with f? values is all < 0.02. 

The results of the hypothesis testing indicate that the lack of university commitment positively 
impacts students’ intention to drop out (H1: B = 0.268; p = 0.000 < 0.01): thus, H1 is accepted. Next, 
degree and course commitment negatively impact students’ intention to drop out (H2: B = -0.1, p = 0.006 
< 0.01); thus, H2 is accepted. Similarly, ineffective time management positively impacts students’ intention 
to drop out (H3: B = 0.172, p-value = 0.000 < 0.01). Thus, H3 is accepted. 

However, the hypotheses H4 and Hd are not supported in this study (B = -0.064; p-value > 10%; 
B = -0.014 > 10%). Additionally, the curriculum design negatively impacts students’ intention to drop out, 
so H6 is accepted (B = -0.088, p-value = 0.079 < 10%). Moreover, the positive instructor feedback does 
not impact students’ intention to drop out, so H7 is rejected. Furthermore, ineffective adaptation to the 
learning environment and low classroom participation all have a positive impact on a student’s intention 
to drop out; thus, H8 and H9 are supported (B = 0.116; p-value = 0.001 < 1%; B = 0.123; p-value = 0.001 
< 0.001). Hypotheses H10, H11, and H12 are not supported in this study. Lastly, personal circumstances 
positively impact a student's intention to drop out; thus, H13 is accepted (B = 0.09; p-value = 0.008 < 1%). 


Table 7 
Differences in dropout intentions by gender, location, and type of university 


Differences between students’ dropout intentions by gender 


(Male)___(Femae)__diference__mean ference 250% 97.50% | PRY” 
DCC -> SDI -0.096 -0.129 0.032 0.002 -0.136 0.151 0.666 
PC -> SDI 0.116 0.053 0.064 -0.002 -0.14 0.136 0.351 
UF -> SDI -0.03 -0.001 -0.029 0.006 -0.162 0.163 0.739 
IS -> SDI 0.095 0.137 -0.043 0.002 -0.146 0.138 0.57 
IALE -> SDI -0.063 0.03 -0.093 0.000 -0.146 0.156 0.226 
SAI -> SDI 0.13 0.117 0.013 0.004 -0.136 0.139 0.837 
PIF -> SDI 0.044 -0.048 0.093 -0.001 -0.166 0.166 0.27 
LUC -> SDI 0.253 0.299 -0.046 0.001 -0.145 0.151 0.549 
LCP -> SDI -0.001 -0.131 0.13 -0.002 -0.181 0.185 0.178 
ITM -> SDI 0.184 0.13 0.054 0.000 -0.135 0.126 0.426 
CD -> SDI -0.149 -0.019 -0.13 -0.002 -0.212 0.203 0.215 
ASA -> SDI 0.017 -0.037 0.054 0.000 -0.161 0.157 0.482 
CSE -> SDI -0.015 0.092 -0.107 -0.005 -0.167 0.169 0.195 

Difference between students’ dropout intentions by area 

suburban) __(Uiben) __diference _meandference 25% 97.50% | STE 
DCC -> SDI -0.051 -0.136 0.085 -0.002 -0.157 0.148 0.267 
PC -> SDI 0.13 0.056 0.074 0.003 -0.133 0.144 0.301 
UF -> SDI 0 -0.009 0.009 0.000 -0.179 0.183 0.928 
IS -> SDI 0.171 0.109 0.062 0.000 -0.15 0.146 0.399 
IALE -> SDI -0.077 0.016 -0.092 -0.004 -0.162 0.15 0.259 
SAI -> SDI 0.149 0.083 0.066 0.002 -0.153 0.151 0.383 
PIF -> SDI | 0.036 -0.035 0.071 0.000 -0.173 0.187 0.443 
LUC -> SDI 0.363 0.235 0.128 0.002 -0.156 0.151 0.118 
LCP -> SDI 0.058 -0.131 0.189 0.010 -0.2 0.211 0.073 
ITM -> SDI 0.115 0.199 -0.084 -0.002 -0.142 0.129 0.256 
CD -> SDI -0.089 -0.065 -0.024 0.001 -0.204 0.218 0.835 
ASA -> SDI -0.023 0.009 -0.033 -0.005 -0.182 0.186 0.686 
CSE -> SDI -0.051 0.059 -0.11 -0.003 -0.171 0.165 0.205 
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Difference between students’ dropout intentions by type of university 
Original Original 


(Public (Private gierence mean dierence 250% 97-50% TN on 
university) university) 
DCC -> SDI -0.068 -0.142 0.073 -0.003 -0.151 0.151 0.359 
PC -> SDI 0.105 0.059 0.045 0.000 -0.135 0.133 0.481 
UF -> SDI 0.037 -0.061 | 0.098 0.008 -0.168 0.171 0.269 
IS -> SDI 0.103 0.131 -0.028 -0.005 -0.157 0.129 0.692 
IALE -> SDI -0.063 0.039 -0.102 0.003 -0.15 0.150 0.207 
SAI -> SDI 0.128 0.127 0.001 -0.003 -0.149 0.139 0.993 
PIF -> SDI -0.016 -0.009 -0.007 -0.002 -0.166 0.159 0.930 
LUC -> SDI 0.262 0.29 -0.029 0.000 -0.154 0.159 0.709 
_LCP -> SDI 70.088 0.068 0002 0.001 0.196 0-189 0843 
ITM -> SDI 0.15 0.196 -0.046 0.003 -0.131 0.128 0.498 
CD -> SDI -0.119 -0.045 -0.075 -0.004 -0.199 0.194 0.474 
ASA -> SDI -0.053 0.126 -0.179 -0.002 -0.166 0.154 0.023 
CSE -> SDI | 0.019 0.007 0.012 0.000 -0.17 0.173 0.895 


(Source: own author) 


Table 7 presents the results of a multigroup analysis examining the differences in students’ intention 
to drop out of school based on three variables: gender (Male, Female), location of activity (Urban, 
Suburban), and type of university (Public, Private). The results of the statistical tests indicate that the 
p-values are all greater than 0.05, suggesting that there is no significant difference in students’ intention 
to drop out of school based on gender, location of activity, or type of university, except for ASA - SDI, the 
access to support from academic advisors (ASA) has a differential impact on students’ dropout intentions 
based on the type of university (p = 0.023 < 0.05). 


Discussion 


The research findings indicate a higher intention to drop out among students who need more 
commitment to the institution, particularly in Ho Chi Minh City. These findings are consistent with previous 
studies such as Willcoxson (2010), Bean (1980). Willcoxson (2010) identified that students are likelier to 
leave university when they lack organizational commitment. Bean (1980) found that students who lack 
commitment to the institution tend to withdraw from the learning process. 

The commitment to credentials and courses inversely impacts students’ intention to drop out. 
Previous studies such as Yorke and Longden (2008), Tinto (2012) have shown that strong commitment 
to academic qualifications and courses is associated with stability and persistence in students’ learning, 
reducing the likelihood of dropouts. 

In addition, ineffective time management positively impacts students’ intention to drop out in 
Ho Chi Minh City. The research findings are consistent with previous studies such as Nieudwoudt and 
Pedler (2021), Willcoxson (2010). Ineffective time management leads to student discouragement and the 
formation of dropout intentions (Nieudwoudt and Pedler, 2021). Students who are unable to balance their 
personal time and study time are more likely to develop intentions to drop out (Willcoxson, 2010). 

The design of the course program has an inverse impact on students’ intention to drop out. The 
research findings align with previous studies as well. Rovai and Jordan (2004) demonstrated that flexibility 
in the curriculum can increase student commitment and reduce dropout intentions. Willcoxson (2010) 
stated that carefully designed and logical courses can be highly effective in education and contribute to 
reducing students’ intention to drop out. 

Low classroom participation by students has a positive impact on their intention to drop out. Some 
previous studies have also shown that students who frequently skip classes and do not participate in 
classroom activities have a positive relationship with dropout intentions (Willcoxson, 2010). Kuh et al. 
(2008) argued that low-engagement students have poorer academic outcomes and higher intentions 
to drop out. Students with low classroom participation need more determination to complete the course 
(Feldman, 1994). Lastly, personal circumstances positively impact students’ intention to drop out in Ho 
Chi Minh City, which is consistent with previous research. Personal circumstances influence students’ 
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decisions to drop out (Bean, 1980; Willcoxson, 2010). 

The remaining factors, such as instructors’ teaching skills and attitude, instructor support, instructor 
feedback, facilities, socio-cultural environment, and access to academic advisors, do not impact students’ 
intention to drop out. These research findings contradict previous studies (Hausmann et al., 2007; Kuh 
et al., 2007; Nieudwoudt and Pedler, 2021; Willcoxson, 2010). When interviewing a group of students in 
various institutions, they expressed the belief that instructors’ teaching skills and attitude do not influence 
their intention to drop out. According to the interviewed student group, effective instruction requires 
instructors to have practical experience, expertise, and in-depth knowledge. Students are concerned 
with teaching methods and the enthusiasm and dedication of instructors. Instructor support helps 
students effectively address difficulties during the learning process. However, access to and support from 
instructors is just one aspect, and if timely support from instructors is not received, students can seek 
assistance from friends to resolve their issues. Whether students receive access to and support from 
instructors does not impact their intention to drop out. The research results indicate that factors such 
as timely feedback, facilities, socio-cultural environment, and access to academic advisors positively 
but statistically insignificantly influence students’ intention to drop out. The research findings may not 
be suitable for the actual situation in Vietnam and cannot be considered as factors affecting students’ 
intention to drop out. 


Conclusion 


Based on the practical context in Vietnam regarding students’ dropout rate and considering the 
research by Willcoxson (2010), the study adjusted and identified factors influencing students’ intention 
to drop out in Ho Chi Minh City. The influencing factors were explored by surveying 804 students from 
public and private universities in suburban and urban areas. These factors include: 1) Lack of commitment 
to the institution, 2) Degree/course commitment, 3) Time management, 4) Course design, 5) Students’ 
ineffective adaptation to the learning environment, 6) Limited classroom participation, and 7) Personal 
circumstances. Additionally, the study found that the following factors did not impact students’ intention 
to drop out in Ho Chi Minh City: Teaching skills and attitude of instructors, instructor support, instructor 
feedback, facilities, socio-cultural environment, and access to academic advisors. Based on the findings, 
the research made two main contributions. 

In terms of theoretical aspects, the research has identified factors that influence students’ intention 
to drop out in Vietnam, specifically in Ho Chi Minh City, where previous studies were scarce. These factors 
align with the practical situation for Ho Chi Minh City students. The study provides a comprehensive 
understanding of the factors influencing students’ intention to drop out. This helps researchers and 
educational experts better understand the factors that may lead to student disengagement or loss of 
interest in learning. Factors such as lack of commitment to the institution, degree/course commitment, 
time management, course design, ineffective adaptation to the learning environment, and personal 
circumstances have been identified to assess students’ intention to drop out. This can assist educational 
managers and instructors develop appropriate support measures and interventions to maintain and 
enhance students’ engagement and academic success. 

In practical terms, the research findings can be used to develop programs and educational policies 
to reduce the student dropout rate. Universities and instructors can implement measures such as 
enhancing student commitment, creating conducive learning environments, improving time management, 
and offering better-designed courses to enhance student engagement and interest in learning. The 
specific results from the study can also be used to propose individual support measures for students. This 
may involve counseling and personal support to help students overcome personal and familial difficulties 
caused by their circumstances. The research also highlights that factor such as teaching skills and attitude 
of instructors, instructor support, and academic advising are not decisive factors in students’ intention to 
drop out in Ho Chi Minh City. This can help universities focus on other aspects of the learning experience 
to create a positive learning environment and better support students. 

The study is limited to the research scope within the urban and suburban areas of Ho Chi Minh 
City. Therefore, expanding the research scope to include universities in other regions of Vietnam may 
be necessary to gain a more comprehensive understanding of the issue. Additionally, the study needs to 
address the financial factors and cost of education. Surveying the impact of financial factors and the cost 
of education could be an essential part of understanding students’ intention to drop out. 
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Appendix 


Student’s dropout intentions (SDI) 
SDI: I often think about dropping out of school. 


SDI2: I am actively seeking job opportunities and alternative learning options, so I may 
leave the university. 


SDI3: There is a possibility that I will drop out of university within the next year. 


SDI4: I am looking for suitable timing to drop out of school. 
Lack of university commitment (LUC) 


LUC1: I am studying at this university as a steppingstone to transfer to another univer- 
sity. 
LUC2: The reputation of my university is very important for job applications. 


LUC3: I am attending this university because I did not meet the requirements of other 
preferences. 


LUC4: I am satisfied with the university I am currently studying at 
LU 


Degree and course commitment (DCC) 
DCC1: I have obvious reasons for studying at this university 


ITM3: It is challenging to balance family responsibilities and university studies. 


CS: I am satisfied with my personal experience at the university. 


ITM4: It is challenging to balance work and university studies. 
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SAI4: The faculty team clearly communicates their expectations from the students right 
from the beginning 


SAI5: The professors always create a sense of closeness with the students. 


SAI6: I have encountered difficulties in understanding the accent of some instructors 
while listening 


Curriculum design (CD) 
CD1: My professors incorporate real-life examples into their teaching curriculum. 
CD2: What I am learning at the university has been researched and proven. 


CD3: I am satisfied with the job experiential opportunities introduced by the university. 


Positive instructor feedback (PIF) 


SAI7: I have had some unpleasant experiences with certain instructors. 

IS4: The faculty team is always available when I need them. 

IS5: My professors genuinely make an effort to understand the difficulties students face 
in their learning process 


LCP3: I appreciate the opportunity to interact with students from diverse cultural back- 
grounds at the university. 


LCP4: When working in groups, I enjoy collaborating with peers from different cultural 
backgrounds. 
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UF2: The library is spacious, well-ventilated, and offers a diverse range of study materi- 
als. 


Access to support from academic advisors (ASA) 


CSES: I feel a sense of belonging to the university community. 
ASAI: I receive good advice from the university regarding career choices 


CSE4: I appreciate the physical facilities and environment of the university campus. 


ASA4: I easily receive assistance when needed from the management team 


ASA2: I receive good advice from a career counselor in choosing a profession for my- 
self. 


ASA3: I receive good advice from the university regarding career choices. 


CSE6: I sometimes feel lonely in the university. 
CSE7: I find it easy to commute to the university. 


PC4: I worry about the accumulating debt while studying at university 
PCS: I have financial issues. 
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Abstract: Previous studies suggest that competition and motivation are reliable predictors of academic dishonesty. 
However, little is known about the role of situational factors in predicting academic dishonesty. Some studies have found that 
online learning is more prone to academic dishonesty, but others have found the opposite. This study focuses on academic 
dishonesty, how it relates to competitive orientation and motivation, and how that differs in two class modes (online vs offline). 
This study was conducted in Indonesia during early 2022, transitioning from online learning due to the Covid-19 pandemic 
to normal-offline learning. A total of 404 university students participated in this study. Most participants (74.2%) reported they 
cheated more frequently in online than in offline learning. The independent sample t-test indicated that students in the online 
learning group showed higher academic dishonesty than students in the offline learning group. Latent regression analysis 
showed that amotivation, hypercompetitive orientation, and learning mode are significant predictors of academic dishonesty. 
These findings imply that transitioning from offline to online learning during the pandemic negatively affected academic integrity. 

Keywords: academic dishonesty, hyper-competition, motivation, online learning. 


Introduction 


The covid-19 pandemic has changed many things, including student learning behaviour. One issue 
that educators and researchers often ask is whether the mode of learning has an impact on students’ 
academic dishonesty. At the beginning of the pandemic, all learning processes had to be switched 
to online. Many researchers have investigated the effect of switching learning to online on academic 
dishonesty (e.g., Erguvan, 2021; Janke et al., 2021; Jose, 2022; Malik et al., 2023). Most studies found an 
increasing number of academic dishonesty when learning mode had to switch to online. Two years after 
the first outbreak, several universities have returned to normal in-person learning. This study aimed to 
examine the academic dishonesty of university students in Indonesia during early 2022, transitioning from 
online learning due to the Covid-19 pandemic to normal-offline learning. At that time, some universities 
had returned to offline learning while others were still conducting online learning. 

Academic dishonesty refers to any behaviour that intentionally violates academic rules for personal 
gain (Janke et al., 2021). This term is often used interchangeably with the term cheating. However, the 
term academic dishonesty was used in this study because it covers a broader range of behaviours, such 
as plagiarism, cheating on exams, or lying. Academic dishonesty is firmly attached to students. Previous 
studies have consistently reported that most students admit to having committed academic dishonesty 
during their studies (Bernardi et al., 2004; Teixeira and Rocha, 2010). These findings have prompted 
researchers to investigate what internal or external factors influence students’ academic dishonesty 
behaviour. 

The effect of learning methods on academic dishonesty has been studied by several researchers 
(for a review, see Holden, Norris and Kuhlmeier, 2021). Some studies found that online learning increased 
the risk of academic dishonesty (Janke et al., 2021; Khan and Balasubramanian, 2012; King and Case, 
2007; Lanier, 2006). Some other studies found that online learning reduced the risk of academic dishonesty 
(Grijalva and Kerkvliet, 2006; Peled et al., 2012), while others found no difference between online and 
Offline learning (Spaulding, 2009; Spaulding, 2009). These findings are not conclusive and, therefore, 
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interesting for further study. 

Several methodological and conceptual reasons explained these different findings. Watson and 
Sottile (2010) suggested that operational differences in academic dishonesty among studies were the 
main cause of inconclusive results. In addition, the context of learning is also essential. For example, 
Janke et al. (2021) suggested that when learning is changed from offline to online due to compulsion 
(e.g., Covid pandemic, which limits face-to-face interactions), the motivation that arises from students 
is extrinsic motivation or even amotivation. It led to high levels of academic dishonesty. Several finding 
supports this claim and found a significant increase in academic dishonesty during the Covid-19 pandemic 
timeframe (Comas-Forgas et al., 2021; Jenkins et al., 2023; Malik et al., 2023). One factor contributing 
to cheating, which is linked to decision-making, is that cheating takes place based on the presence of 
opportunities (Adzima, 2020). The matter of opportunity has gained increased attention, especially with 
the expansion of online education within higher education. From the perspective of the theory of planned 
behaviour (TPB, Ajzen, 1991), perceived control behaviour is a crucial aspect that encourages people to 
cheat. When students perceive a sense of anonymity and a lack of sufficient monitoring in an online class, 
they might believe that the chances of getting caught while cheating are low, leading to a higher likelihood 
of engaging in academic dishonesty (Kajackaite and Gneezy, 2017). However, if learning is conducted 
online because of the student's choice, intrinsic motivation emerges, which reduces academic dishonesty. 
In this context, replication needs to be performed to examine the impact of these situational factors. 
Nowadays, a combination of in-person and online classes is more prevalent without any compulsion. The 
findings of this study could be a basis for making policies related to online learning. 

Individual differences in internal factors are also essential for predicting academic dishonesty. 
Internal factors to be investigated further in this study are motivation and competitive orientation. In 
literature, competition has been associated with academic dishonesty. For example, Taylor, Pogrebin 
and Dodge (2002) stated that the pressure of competition encourages dishonest behaviour in order to 
get the best grades. Two dimensions of competitive orientation influence academic dishonesty most: self- 
development and hypercompetitiveness (Orosz, Farkas and Roland-Lévy, 2013). Self-development refers 
to self-growth, not considering competitors as enemies, and enjoying and learning from the competition 
process. In contrast, hypercompetitive individuals try to win at any cost. They see their competitors as 
enemies and can be aggressive towards them. Self-development is assumed to correlate negatively with 
academic dishonesty, while hypercompetitiveness positively correlates with academic dishonesty. 

Motivation is also consistently cited in the literature as a predictor of academic dishonesty (Krou, 
Fong and Hoff, 2021; Orosz, Farkas and Roland-Lévy, 2013). Vallerand et al. (1992) explained that 
intrinsic motivation arises when individuals engage in activities for their own sake and for the satisfaction 
that comes from it. Extrinsic motivation occurs when individuals engage in activities to achieve goals, not 
for their own sake. When individuals do not feel causality between their actions and the results, this can 
be labelled as amotivation. Individuals with amotivation have neither extrinsic nor intrinsic motivation and 
usually feel incompetent. The role of motivation in predicting academic dishonesty was briefly summarized 
in a meta-analysis study conducted by Krou, Fong and Hoff (2021). Academic dishonesty correlated 
negatively with intrinsic motivation and positively with extrinsic motivation and amotivation. 


Study objectives 

Most previous studies investigated internal and situational factors separately in predicting academic 
dishonesty. However, academic dishonesty cannot be separated from these two factors. Therefore, this 
study aimed to investigate the role of internal factors (i.e., motivation and competitive orientation) and 
situational factors (i.e., learning mode) in predicting academic dishonesty simultaneously. This study was 
conducted during the transition period from online learning due to the Covid-19 pandemic to normal-offline 
learning. With the varied learning mode in Indonesia at that time, this research can contribute scientifically 
to answering disagreements over previous research results. In addition, this research is also helpful in 
providing practical considerations for formulating policies to reduce academic dishonesty. 

In the first step, we established a measurement model for all constructs and analyzed their 
interrelations. The present study intended to resolve methodological issues in previous studies by 
investigating the measurement invariance of the instrument across learning modes (online vs offline). In 
the next step, we conducted latent regression analyses to predict academic dishonesty with motivation, 
competitive orientation, and learning mode as predictors. Specifically, our research questions were as 
follows: 

1. Do motivation and competitive orientation explain students’ academic dishonesty? 

2. Does learning mode explain students’ academic dishonesty beyond motivation and competitive 
orientation? 
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Based on previous research (Krou, Fong and Hoff, 2021; Orosz, Farkas and Roland-Lévy, 2013), 
we hypothesized that academic dishonesty correlates negatively with intrinsic motivation and positively 
with extrinsic motivation and amotivation. In addition, self-development is hypothesized to correlate 
negatively with academic dishonesty, while hypercompetitiveness positively correlates with academic 
dishonesty. Regarding the context of the learning mode, since online learning was conducted due to 
compulsion, we hypothesized that students in online learning have higher levels of academic dishonesty, 
in line with the previous study (Janke et al., 2021). 


Materials and Methods 


Participants 

Participants of this study were active students who took part in both online and offline learning. A 
total of 404 students (55% were women) participated in this study. Participants ranged from 18 - 47 years 
old (M = 21.41, SD = 3.41). Participants consisted of undergraduate students (89%), Master’s students 
(7%), and Doctoral students (3%). A total of 193 students (48%) took offline learning, and 211 (52%) 
took online learning. Data was collected using an online survey in early 2022, where learning modes still 
varied. Some students took online classes, while others attended face-to-face classes. 


Instruments 

Academic dishonesty questionnaire 

The academic dishonesty questionnaire is a measure specifically developed for this study. This 
questionnaire consists of 11 items, which are behavioural indicators indicating academic dishonesty 
relevant to online and offline learning situations. Participants were asked to rate how often they did the 
following activities this semester (e.g., “Copying material from the internet, books, or articles without citing 
the source” or “Making up false excuses for being late to turn in assignments’). Participants answered 
with a response scale ranging from 1 (never) to 5 (always). Please see the appendix for detailed items. 
Cronbach's alpha for this measure was 0.77. After being presented with the questionnaire, participants 
were given one question read as follows: “In which learning mode do you do the activities mentioned 
earlier, online or offline?”. 


Academic Motivation Scale (AMS) 

AMS is a multidimensional scale for measuring three dimensions of motivation: intrinsic motivation, 
extrinsic motivation, and amotivation. This scale was developed by Vallerand et al. (1992). Natalya (2018) 
translated the scale into Indonesian, validated it and made a short-form version. The instructions read 
as follows: “Why do you go to college?”. This scale consists of 15 items. Intrinsic motivation consists 
of seven items, with the following sample item: “Because | experience pleasure and satisfaction while 
learning new things”. The extrinsic motivation subscale consists of six items, with the following sample 
item: “In order to obtain a more prestigious job later on”. The amotivation subscale consists of two items, 
with the following sample item: “Honestly, | don’t know; | really feel that | am wasting my time in school’. 
Participants rated all items on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly 
agree). For detailed items, readers are encouraged to read the original paper by Natalya (2018) . The 
Cronbach’s alpha reliability of these items was 0.86 for intrinsic motivation, 0.83 for extrinsic motivation, 
and 0.76 for amotivation. 


Multidimensional Competition Orientation Inventory (MCOl) 

MCOI is a multidimensional scale to measure the four competitive orientations. This scale was 
developed by Orosz et al. (2018). We translated the scale from English into Indonesian using the 
translation back-translation method. Only two subscales were used in this study: self-development and 
hypercompetitive, as previous findings indicated that these two dimensions were related to academic 
dishonesty. Self-development subscale consists of three items, with the following sample item: “Competitive 
situations allow me to bring the best out of myself’. The hypercompetitive subscale consists of three 
items, with the following sample item: “The most important is winning, no matter what”. Participants rated 
all items on a 6-point Likert-type scale ranging from 1 (Not true of me at all) to 6 (Completely true of 
me). Readers are encouraged to read the original paper by Orosz et al. (2018) for detailed items. The 
Cronbach’s alpha reliability for the self-development was 0.84 and 0.68 for hyper-competitive. 
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Procedures 

Participants were recruited using various strategies, including social media advertisements (e.g., 
Facebook, Instagram, WhatsApp). Participants who were willing to participate in the study completed 
the online survey programmed in PsyToolkit (Stoet, 2017). Participants received no monetary incentives 
for participating in this study. After reading the research description, participants gave their consent to 
be able to move to the questionnaires. All participants gave their informed consent for inclusion before 
participating in the study. The study was conducted following the Declaration of Helsinki. 


Data analysis 

Confirmatory Factor Analysis (CFA) and latent variable regression analysis were performed using 
the ‘lavaan’ package (Schermelleh-Engel, Moosbrugger and Muller, 2003) in the R program (R Core 
Team, 2012). First, the measurement model of all constructs was specified and tested using CFA. The 
measurement invariance of the instruments across learning modes (online vs offline) was investigated to 
ensure that the instruments used in this study have similar meanings for the two groups. Three levels of 
measurement invariance were investigated: configural, metric, and scalar. At the configural level, loadings 
and intercepts were freely estimated. At the metric level, loadings were constrained to be equal across 
learning modes, and the intercepts were freely estimated. At the scalar level, both loadings and intercepts 
constrained to be equal across learning modes. Measurement invariance analysis is met if: ACFI < -.01, 
ARMSEA < .015, ASRMR < .010 (Chen, 2007) for each level of the model. 

Following the CFA, statistic descriptive and intercorrelation among variables were examined then. 
We compared the score of all variables between the online and offline groups using an independent 
sample t-test. Finally, we conducted latent regression analyses to predict academic dishonesty with 
motivation, competitive orientation, and learning mode as predictors. There were two models specified: 
1) model with internal factors as predictors and 2) model with internal factors and situational factors as 
predictors. 

In CFA and latent regression analyses, we used weighted least squares means and variance 
adjusted (WLSMV) estimation because it is better suited to ordinal data (Beauducel and Herzberg, 
2006). We evaluated model fit using several fit indices, including the x’, comparative fit index (CFI), the 
standardized root-mean-squared residual (SRMR), and the root mean squared Error of approximation 
(RMSEA). The following parameters were used to assess the models’ adequacy: CFI >.90, TLI > 0.90, 
SRMR.10, and RMSEA.08 were deemed adequate, and CFI >.95, TLI > 0.95, SRMR.05, and RMSEA.05 
were considered an excellent fit (Schermelleh-Engel, Moosbrugger and Muller, 2003). 


Results 


Measurement model of the instruments 

CFA was performed to examine whether the measurement model for all variables studied fit the 
data. The results of the initial analysis showed that the model had y? = 751.69, df = 449, p <0.01, CFI = 
0.951, TLI = 0.946, RMSEA [90% CIlJ= 0.044 [0.039, 0.049] and SRMR = 0.071. Although the initial model 
had shown satisfactory indicators in terms of CFI, TLI, RMSEA, and SRMR values, the academic honesty 
measurement model had one item with a very low loading factor (A < 0.30). The model was then modified 
by removing that item. The final measurement model had y? = 708.87, df = 419, p < 0.01, CFI = 0.953, 
TLI = 0.948, RMSEA [90% Cl]= 0.046 [0.040, 0.051], and SRMR = 0.071. This final model displayed 
an acceptable fit. The omega reliability (w) of each variable was 0.78 for academic dishonesty, 0.56 for 
amotivation, 0.89 for extrinsic motivation, 0.83 for intrinsic motivation, 0.68 for hypercompetitive, and 0.85 
for self-development. 


Measurement invariance 

Measurement invariance analysis was performed to examine whether the measurement model 
applies equally to groups of students with online and offline classes. Measurement invariance has 
several levels. If the metric invariance is met, then the regression analysis comparison can be performed 
between groups. If the scalar invariance is satisfied, then the latent means can be compared across 
groups meaningfully. The summary of the results of the invariance analysis can be seen in table 1. Table 
1 indicates that the model accuracy indices (i.e., y?, CF], RMSEA, SRMR) of each model are almost 
unchanged compared to other model fit indices. All three models (M1 to M3) did not show a significant 
decrease in model fit, indicating that all constructs achieved scalar invariance between the online and 
offline groups. Therefore, the variable measurement model proved invariant between groups of learning 
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modes, and the latent means could be compared meaningfully across learning modes. 


Table 1. 
Invariance of measurement models based on learning mode groups 
Model Model fit Model comparison 
x2 (df) CFl RMSEA SRMR~ ACFI ARMSEA  ASRMR 
Configural 914.74 (838) 987 021 077 


_Metric(M2) 994.56 (863) 978 ~— 028 079 0.000 (0.006 0.002 
Scalar (M3) _ 1021.26 (888) 978 ~—_—.027 080 0.000 ~_(0.000 0.001 


Descriptive statistics of studied variables 

Participants generally reported relatively low academic dishonesty levels (M = 1.76, SD =0.51). 
Table 2 shows that motivation and competition orientation do not differ in online and offline groups. 
However, academic dishonesty in the two groups was statistically significant, with students in online 
classes having higher levels of academic dishonesty than students in offline classes. The Pearson’s 
correlation analysis among variables indicated that academic dishonesty was positively correlated with 
amotivation and hypercompetitiveness and negatively correlated with extrinsic motivation. Surprisingly, 
there was no correlation between academic dishonesty and internal motivation. For the single question 
about the tendency to cheat in two learning modes, as many as 300 participants (74.2%) reported 
committing academic dishonesty when learning online. 


Table 2. 
Descriptive statistics and intercorrelation of studied variables 
Online — Offline | | | 
Variable (N=211) | (N=193) ¢ i ? : ° 
Mean (SD) Mean (SD) 
1 Ease 5.66 (0.93) 5.66 (1.06) 0.01 
motivation | 
Ne 5.65 (1.05) 5.62(1.15) 0.03 57% | 
motivation a ee oe 
Amotivation 1.88 (1.05) 1.95(1.04) 0.06 -0.35** = -.19*** 


_Hypercompetitive 2.79(1.07) 277(1.08) 002 10 .22"* 13" 32" 
Academic | 
dishonesty | eee 


Note: * p < 0.01, *** p< 0.001. 


On wo 


(1.05) (1.15) 

(1.05) 1.95 (1.04) | | 
Self-development 4.52 (0.89) 446 (1.12) 0.05 53*** fo) Niele | al 

(1.07) (1.08) 

(0.50) (0.52) 


1.84 (0.50 0.28% -19%* -04 31" = -10 | 25" 


Motivation, competitive orientation, and learning mode as predictors of academic dishonesty 

Two latent regression model was performed. In the first model, we regressed academic dishonesty 
on extrinsic motivation, intrinsic motivation, amotivation, self-development, and hypercompetitive. In the 
second model, we added learning modes as the sixth predictor. The results for both models are shown 
in Table 3. 

In model 1, only amotivation and hypercompetitive orientation were significant predictors of 
academic dishonesty. Both amotivation and hypercompetitive have positive effects on academic 
dishonesty. All predictors together explained 25% of the variance in academic dishonesty. In model 
2, the learning mode significantly affected academic dishonesty. Adding learning mode as a predictor 
increased the variance explained to 28%. The negative effect size indicated that students in the online 
class have higher academic dishonesty than students in the offline class. To summarize, in line with our 
assumptions, internal and situational factors were related to academic dishonesty. However, contrary to 
our expectations, only amotivation and hypercompetitive were related to academic dishonesty, while the 
effect of intrinsic-extrinsic motivation and self-development were not significant. 
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Table 3. 
Regression of academic dishonesty on motivation, competitive orientation, and learning modes 
Predictor Model 1 Model 2 
B SE B B SE 8 
Extrinsic motivation -0.01 0.11 -0.01 0.02 0.12 0.01 
Intrinsic motivation 0.01 007 001 -001 O07 001 
Amotivation 0.47 0.11 041 050 012 0.42" 


Self-development -0.14 0.08 -0.12 0.15 0.08 -0.13 

Hypercompetitive 0.24 0.07 0.20** 0.25 007 021 
Learning mode - | - 0.21 005 = -0.18""* 

R? 0.25 0.28 


y2(df) 708.87 (419) 735.65 (444) 
CFI/TLI 0.953 / 0.948 0.953 / 0.947 
RMSEA/ SRMR 0.041 /0.071 0.040 / 0.070 


Note: *** p < 0.001, B=unstandardized coefficient, SE = Standard Error, 8 = standardized coefficient, learning mode 
was coded as 0 for online and 1 for offline class. A negative regression coefficient indicated that online classes showed higher 
academic dishonesty. 


Discussions 


This study aimed to examine whether internal factors (i.e., motivation and competitive orientation) 
and situational factors (i.e., learning mode) predict academic dishonesty. The main findings in this study 
indicate that motivation, competitive orientation, and learning mode contribute to academic dishonesty. 
Specifically, amotivation and hypercompetitiveness are two internal factors that significantly influence 
academic dishonesty. Amotivation, characterized by a lack of interest in academic pursuits, can lead 
students to resort to unethical practices to circumvent their disinterest in learning (Deci and Ryan, 
1985). Research has demonstrated that amotivated students are more likely to cheat, perceiving these 
behaviours as shortcuts to coping with academic responsibilities (Murdock, Hale and Weber, 2001). On 
the other hand, hypercompetitiveness, which reflects an intense desire to outperform others at any cost, 
is associated with a higher propensity for academic dishonesty. In academic contexts, hypercompetitive 
individuals may view academic success as a zero-sum game, leading them to resort to unethical actions to 
gain a competitive edge over their peers, even if it involves undermining their fellow students (Anderman 
and Danner, 2008). In addition, the learning mode also plays a role; students who study online tend to 
show higher academic dishonesty. Most students also reported committing academic dishonesty when 
learning online. However, it is important to note that not all students engage in academic dishonesty. 
Some might even have higher levels of academic integrity in online learning environments. 

Our findings provide evidence to resolve the debate about whether learning modes contribute 
to academic dishonesty. The results of this study support several previous researchers who found that 
dishonesty in online learning is higher than in offline learning (Janke etal., 2021; Knan and Balasubramanian, 
2012; King and Case, 2007). Several reasons explain the inconsistency of previous findings, including 
methodological reasons. The instruments used as comparisons can have different operations, which 
may also be irrelevant for the two types of learning modes. For example, the indicator “Participate in 
class while doing other activities during learning” may be more easily agreed upon by respondents who 
are in online classes. In this study, we first tested for measurement invariance of the instruments before 
comparing academic dishonesty between online and offline groups. Thus, the instrument used has the 
same meaning and is not biased when used to compare the two groups. 

Another explanation regarding the inconsistency of previous findings is explained by Janke et al. 
(2021). They stated that when learning is changed from offline to online due to compulsion (e.g., Covid 
pandemic, which limits face-to-face interaction), the motivation that arises from students is extrinsic 
motivation or even amotivation. This has led to high levels of academic dishonesty. This explanation 
seems relevant to the findings of this study because students who take online classes are mostly out of 
compulsion. 

Several studies also suggested that the theory of planned behaviour (TPB, Ajzen, 1991) can explain 
academic dishonesty in online learning (Ababneh, Anmed and Dedousis, 2022; Chudzicka-Czupaia et al., 
2016). TPB proposes that people’s attitudes, subjective norms, and perceived behavioural control shape 
their intentions and, ultimately, their behaviour. Students may perceive online learning as less important 
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or less rigorous, or they may be concerned about the quality of instruction. It results in a low attitude 
towards online learning, which can contribute to a greater willingness to engage in academic dishonesty. 
The academic norms may be different in online learning. Students may feel less social pressure to behave 
honestly in an online setting. Finally, students’ perceived behavioural control in online learning may be 
influenced by factors such as distractions in the home environment or technical difficulties. These factors 
can contribute to a greater lack of control, increasing the likelihood of cheating. 

Of the five predictors in the model, only two played a significant role: amotivation and 
hypercompetitiveness. Although this finding aligns with previous findings (Krou, Fong and Hoff, 2021; Orosz, 
Farkas and Roland-Lévy, 2013), it is still intriguing because previous research examined motivation and 
competitive orientation separately. We examined motivation and competitive orientation simultaneously. 
These findings show two unique faces of “cheaters”. Academic cheaters have two possibilities. First, they 
have low motivation to study, or second, they consider excessive competition, so they want to justify any 
means to win. 


Limitations 

This research has several limitations that need to be considered. First, we only differentiated 
online and offline groups. We classified students who took blended learning as offline because they had 
started face-to-face learning. However, students with blended learning might have different time frames 
to reflect on themselves when completing surveys. Second, this study is limited to a sample of university 
students. University students are assumed to be more familiar with technology than elementary-high 
school students. Thus the generalization of this research for elementary-high school students may be 
different. Third, the study relied on self-reported measures of academic dishonesty, which may be subject 
to social desirability bias. Future studies could use more objective measures of academic dishonesty, 
such as plagiarism detection software. 


Conclusions 


Overall, this study contributes to the growing body of literature on academic dishonesty and its 
predictors. This research shows that the higher the amotivation and hypercompetitive orientation, the 
greater the tendency for students to commit academic dishonesty. In addition, the learning mode also 
plays a role; students who study online tend to show higher academic dishonesty. Most students reported 
they cheated more frequently in online than in offline learning. 

This research has several implications for the practice of education in universities. First, face-to- 
face learning is still essential not only for the transfer of knowledge but also for the transfer of values. 
Educators need to adjust the design of online learning as a proactive method to reduce academic 
dishonesty. Second, educators are encouraged to implement interventions that promote student motivation 
and reduce hypercompetitive orientation, such as using collaborative learning. Finally, more research 
is needed to better understand the complex relationship between motivation, competitive orientation, 
learning mode, and academic dishonesty in different cultural contexts. 


Appendix 


Academic dishonesty questionnaire 

Please indicate how often you do the following activities / Tunjukkan seberapa sering Anda 
melakukan halhal ini. 

1 = never / tidak pernah, 2 = rarely / jarang, 3 = sometimes / kadang-kadang, 4 = often / sering, 5 
= always / selalu 

1. Logging into a course and engaging in other activities during course time / Ikut kelas sambil 
mengerjakan kegiatan lainnya selama pembelajaran 

2. Solving tasks together with other students that were meant as individual assignments / 
Mengerjakan tugas dengan teman lainnya meskipun itu adalah tugas individu 

3. Writing references in papers even though | have never read the references / Menulis referensi di 
makalah yang saya tulis meskipun referensi tersebut tidak pernah saya baca 

4. Copying material from the internet, books, or articles without writing down the source / Menyalin 
materi dari internet, buku, atau artikel tanpa menuliskan sumbernya 

5. Having others do individual assignments and handing them in as own work / Meminta tugas 
individu milik teman untuk saya kumpulkan sebagai pekerjaan saya 
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6. Making false excuses for being late in submitting assignments / Membuat alasan palsu karena 
terlambat mengumpulkan tugas 

7. Letting someone else sign a course attendance sheet to cover up not being present in the course 
/ Meminta teman untuk mengabsenkan saya saat saya tidak hadir dalam pembelajaran 

8. Trying to bribe an instructor to get deadline extensions or better grades / Merayu dosen agar 
mendapat tambahan waktu deadline atau nilai yang lebih baik 

9. Paying others to do my own learning assignments / Membayar orang lain untuk mengerjakan 
tugas saya 

10. Solving exam questions by using additional materials or the internet without permission / 
Menjawab pertanyaan ujian dengan mencari materi di internet tanpa izin 

11. Exchanging ideas with others about possible answers during an examination / Berdiskusi 
dengan teman tentang kemungkinan jawaban pertanyaan ujian 
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Abstract: Virtual Reality technologies have been established as an effective tool for the treatment of a variety of mental 
health disorders. Despite those positive results, it remains unclear whether these findings can generalize to the healthy population. 
In the current study, we assessed to what degree a virtual scenario consisting of a tropical beach can be successfully applied 
for the reduction of state anxiety and negative mood in healthy individuals. The study was performed on 50 volunteers aged 
18 to 45 that were individually exposed to virtual reality. Before and after the virtual reality session, the emotional state of the 
participants was measured through questionnaires. Using linear mixed effect models, we obtained evidence for the effectivity 
of a single virtual reality session for negative state reduction, namely, we observed a decrease in state anxiety and negative 
mood. Additionally, a positive mood of participants increased after the session, however, no statistical evidence was obtained. 
The results confirm our hypothesis that state anxiety and negative mood can be decreased using a low-cost virtual reality 
device. Based on our findings, we suggest that virtual scenarios can improve the emotional well-being and the quality of life of 
the general population. 

Keywords: virtual reality, natural scenarios, state anxiety, negative and positive mood, sense of presence. 


Introduction 


Virtual reality (VR) technologies are becoming increasingly involved in society and the process of 
human life. The area of mental health is not an exception in the 21* century trends, and VR has started to 
become a common tool for the study, prevention, assessment, and treatment of psychological disorders 
(e.g., Glanz, Rizzo and Graap, 2003; LaValle, 2023; Mancuso et al., 2023). In particular, VR technologies 
have been successfully applied to treat various mental health problems such as anxiety disorders (Carl et 
al., 2019; Opris et al., 2012; Parsons and Rizzo, 2008), eating disorders (Clus et al., 2018; Marco, Perpina 
and Botella, 2014), addictions (Segawa et al., 2020) or as a non-pharmacological analgesic procedure to 
reduce acute pain and anxiety (Kenney and Milling, 2016; Triberti, Repetto and Riva, 2014). 

Nowadays, the most established use of VR is as an exposure-technique to treat anxiety-related 
disorders (Freeman et al., 2017). Research studies have shown its clinical efficacy to treat specific 
phobias (Garcia-Palacios et al., 2002; North, North and Coble, 1998), the Posttraumatic Stress Disorder 
(PTSD; Beidel et al., 2019; Rothbaum et al., 2003) or panic disorders (Botella et al., 2007), among others. 
Although various studies have comprehensively studied the possibilities of VR to treat anxiety disorders, 
it becomes challenging to translate these findings to a healthy population. We, therefore, conducted 
the study to assess to what degree a low-cost VR system can be used for decreasing state anxiety 
and negative mood in the healthy population. We argue that the application of VR technologies could 
substantially help in reducing the anxiety levels and in promoting a positive mood in healthy individuals. 

In the rest of the introduction, a historical overview of the development of VR technologies will be 
provided. Then, the problem of human involvement in VR will be discussed. Next, an overview of the 
research on anxiety disorders by utilizing VR technologies will be presented. At the end of the introduction, 
the goals of the current study and the hypotheses will be listed. 
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Historical overview 

The modern VR started its early development in the 1950s and 60s accompanied by several key 
inventions. This included patenting the Sensorama in 1957 by M. Heilig. The goal of the Sensorama 
was to use the senses of the user including components such as odor generators and vibrating chairs 
while aiming to provide a complete multisensory experience (Heilig, 1962). Following this in 1961, the 
Headsight, the first supervisor of head-mounted displays that included motion tracking and dual monitor 
displays, was designed by Philco Corporation (Peddie, 2017). Sutherland (1965) designed the Ultimate 
display, that innovatively used the computer interface. This allowed users to interact using a simple virtual 
scenario in real time frame. 

The formalization of VR occurred in 1989 with J. Lanier coining the term “virtual reality’. At that 
point, VR has been extensively used in research and psychiatric treatment (Lanier, 1992). In the decades 
following that, VR started to be used by mental health professionals as an exposure method to treat 
anxiety disorders. The effectiveness of VR-based Exposure Therapy (VRET) was evaluated in the primary 
research with the focus on the treatment of acrophobia (Rothbaum et al., 1995). Based on this study, 
VRET was proved to be effective and that led to a number of follow-up studies on the use of VR-based 
therapy for the treatment of anxiety disorders and, in general, other mental health problems. 


The problem of human involvement in virtual reality 

An important question for the application of VR is the problem of human involvement in virtual 
scenarios. This involvement was found to be associated with the formation of a genuine emotional 
attachment of participants to virtual characters (Gould, Coulson and Howard, 2012). Slater (2009) 
discussed the Place Illusion (PI) and Plausibility Illusion (Psi) as the predictors of individuals’ realistic 
perception of virtual environments. Continuing this idea, Pillai, Schmidt and Richir, (2013) framed the 
concept of the psychological illusion of presence while being in an alternative reality. This concept was 
understood as a continuous cognitive process that concerns perceived objects and phenomena while the 
mind functions and perceives in the same way as in the real world. The authors suggested calling it Self- 
Evoked Reality, an endogenously induced reality that is a component of the general model of reality. They 
adhered to the position that the psychological alternative reality can be interrupted leading to the person 
returning to the reality. Before this critical moment, most vivid sense of presence is experienced that is 
accompanied by corresponding emotional experiences. The idea of psychological VR and the possibility of 
it being interrupted indicate broad prospects for the treatment of mental disorders, psychological trauma, 
loss and other problems. 

The ability to simulate real situations in a virtual environment drew the attention of the researchers to 
the possibilities of its use for the study, assessment, prevention, and treatment of mental health disorders. 
Since then, VR has been successfully applied to various mental health disorders (e.g., Fornells-Ambrojo 
et al., 2008; Freeman et al., 2010; Riva et al., 2004). These studies opened perspectives to determine 
the essence of the manifestation of thoughts of persecution, the possibility to separate the real world 
from the fantasy world in people with psychopathology. In this direction, VR has established itself as a 
implementation technique for a broader study of the causes and nature of the manifestation of certain 
symptoms, which allows adjusting the therapeutic lines of care for people with certain mental health 
problems. 


Research on anxiety disorders and VR 

For two decades various experimental studies have been conducted to study the effect of VR on 
the treatment of subclinical fears and anxiety disorders. In the first controlled application of VR for the 
treatment of psychological issues, VR therapy was successful in reducing the fear of heights (Rothbaum 
et al., 1995). The result was evaluated by indicators of anxiety, avoidance, attitude, and stress. Significant 
differences in the groups were found for all indicators, so the VR group improved significantly after 
treatment, but the control group did not change. Nowadays, a large body of literature validates the use of 
VR as an exposure technique to treat anxiety-related disorders. Note that recent review and meta-analytic 
studies have shown that VR can be more efficient than imaginal exposure and can have the same efficacy 
as in-vivo exposure (Carl et al., 2019; Meyerbroker and Emmelkamp, 2010; Morina et al., 2015; Morina 
et al., 2015; Parsons and Rizzo, 2008; Powers and Emmelkamp, 2008). In addition, the use of VR may 
increase treatment adherence by reducing the treatment desertion. 

However, beyond the treatment of anxiety disorders through reducing negative emotions, VR has 
been also effectively used in inducing positive emotions. In a recent study that utilized VR technologies, 
positive emotions such as relaxation and joy have been successfully induced while an anxiety level was 
reduced in healthy population (Bafios et al, 2008). With regard to a clinical population, a pilot study, 
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involving 25 patients with Generalized Anxiety Disorder (GAD), was conducted to assess the efficacy of 
a VR system designed to induce relaxation (Repetto et al., 2013). Initial results suggested the efficacy of 
VR and biofeedback devices in reducing stress and anxiety in a clinical population. However, the authors 
also suggested the need to use physiological data to modify specific features of the virtual environment 
in real time. 

In recent VR applications, a variety of relaxation techniques have started to be used in conjunction 
with biofeedback devices. Thus, the anxiety treatment has been proposed to improve through the use of 
a biofeedback virtual reality system (VR), used both for relaxation and for the controlled effects of VR in 
psychiatric treatment (Anderson et al., 2005; Anderson, Rothbaum, and Hodges, 2003). Moreover, in a 
study developed within the EU-funded INTREPID research project, Riva et al. (2009) designed a mobile 
VR system with biofeedback techniques to improve the traditional treatments for GAD. 


The feeling of presence and anxiety level 

One of the principal mechanisms underlying the use of VR as an exposure technique is the feeling 
of presence. In the specialized literature, the construct of presence is usually considered the principal 
mechanism by which VR is effective as a psychological intervention tool (Alsina-Jurnet and Gutiérrez- 
Maldonado, 2010). The term “presence” is usually defined as the “sense of being there” as a part of a 
simulated environment (Steuer, 1992). One of the main results of this illusion is that a virtual scenario 
may evoke the reactions, emotions, and thoughts similar to the experience in a real world (Hodges et al., 
1299). 

The sense of presence in VR can be facilitated by the application of technologies including touch 
gloves, head displays, vibroactive platforms and synthesized sounds that together allow to simulate the 
users’ sensorial channels and actively exploration of the virtual environment. In addition, some virtual 
reality systems are programmed to respond in real time to the actions of the participants. Due to this 
dynamic interaction the more natural and intuitive connection to the virtual reality can be experienced. 
The recent studies identified three main factors that comprise the “sense of presence” construct: realness, 
spatial presence, and involvement (Lessiter et al., 2001; Schubert, Friedmann and Regenbrecht, 2001). 

The relationship between the anxiety level and feeling of presence in both clinical and non- clinical 
sample has been examined in various VR applications (Alsina-Jurnet et al., 2011; Krijn et al., 2004: 
Regenbrecht, Schubert and Friedmann, 1998; Robillard et al., 2003; Schuemie et al., 2000). Although in 
some of these studies the relationship between anxiety level and feeling of presence has been established 
(Schuemie and Bruynzeel, 2000), others failed to correlate these values (e.g., Regenbrecht, Schubert and 
Friedmann, 1998). One of the possible explanations of these contradictory results is the differences in the 
sample: while some studies use a clinical population, others draw conclusions on the healthy individuals. 
The mentioned limitations make it difficult to summarize the current findings. A continuing challenge is, 
therefore, to establish whether there is a relationship between anxiety level and feeling of presence in 
healthy individuals. 

To summarize, many of the studies utilizing VR for the treatment of mental health problems were 
aimed to treat anxiety disorders. Thus, the results showed that VR use led to significant reduction in 
anxiety symptoms. Additionally, its effectiveness (or was more effective) was similar as compared to 
traditional exposure interventions. Moreover, it had a powerful effect the persons’ day-to-day life. Despite 
those findings, it remains unclear whether VR technologies can affect the reduction of anxiety and 
negative emotions in the general population. Thus, the goal of the present study was to investigate with a 
non-clinical sample the efficacy of virtual experience for reducing negative state conditions and inducing 
positive ones. 

In the current study, we have the following hypotheses: 

1) State anxiety and the negative mood can be reduced utilizing a low-cost VR system. 
2) Positive mood can be induced utilizing VR technologies 
3) There is a negative relationship between levels of state anxiety and feeling of presence 


Materials and Methods 


Participants 

The sample consisted of 50 participants (37 females and 13 males), students of Armenian-Russian 
University. The mean age of participants was 22.34 (SD = 5.28, age range = 18-45). All participants were 
healthy individuals with no history of neurological and psychiatric disorders. 

Before their participation, they gave informed consent in the form of a written statement in 
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accordance with the Declaration of Helsinki. In this statement, they agreed that participation in the study 
was voluntary, individually conducted and had no negative consequences, that the data obtained in the 
course of this study would be processed anonymously. 


Questionnaires 

The following questionnaires were administrated: 

* STAI (State-Trait Anxiety Inventory, Form X, S-Anxiety scale; Spielberger, 2010). A questionnaire 
that assesses state anxiety level. This sub-scale comprises 20 items (e.g., | am presently worrying over 
possible misfortunes) graded on a Likert scale from 0 (not at all) to 3 (a great deal). 

* SUDS (Subjective Units of Discomfort Scale; Wolpe, 1990). Participants evaluate their level of 
anxiety on a scale of 100 points where 0 means no anxiety and 100 - extreme anxiety. 

* PANAS (Positive and Negative Affect Schedule; Watson, Clark and Tellegen, 1988). A self-report 
questionnaire consisted of 20 adjectives which are related to 10 positive emotions (positive affect) and 10 
negative emotions (negative affect). Participants were asked to define the degree they felt the emotions 
at the current moment using a five-point Likert scale where 1 was ‘not at all’ and 5 — ‘extremely’. 

* IPQ (Igroup Presence Questionnaire; Schubert, Friedmann and Regenbrecht, 2008). A 
questionnaire invented to evaluate the sense of presence in VR. This questionnaire consists of 14 items 
that need to be evaluated using a seven-point Likert Scale. Three subscales rating different dimensions of 
presence are included in IPQ. The Spatial Presence sub-scale reflects the sense of being physically inside 
the virtual reality environment, Realness sub-scale estimates the sense of reality attributed to the virtual 
environment, while the Involvement evaluates the attention devoted to the virtual reality environment. 
Additionally, a general item that estimates the “sense of being here” is included in IPQ. 


Hardware 

The virtual scenario was displayed by a standalone VR glasses (Pico Interactive Goblin VR) with 
a resolution of 2560 x 1440 pixels, 92° field of view, and a screen refresh rate of 70 Hz (<20 ms) (see 
Figure 1). 


Figure 1. Standalone Pico Goblin VR headset 


Software 

RelaxVR (https://www.relaxvr.co/): An App that combines the exposure to different 360° videos 
of natural environments (including beaches, forests, rivers, waterfalls, among others) with guided 
meditations, and soothing music to accompany the visual and meditative experience. 

For the present study, researchers used the video of a beach located in Tasmania (Australia) (see 
Figure 2) with an embedded Russian narrative consisting of a 5-minute breath awareness technique. Note 
that the breath technique was specifically designed and recorded for the present study. 


oo . ~_ 


a og 


Figure 2. A screenshot of the virtual beach used in the present study 
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Procedure 

A within-subjects design with two evaluation moments (pre- test and post-test) was used. The 
duration of each experiment was approximately 30 min, and all participants did it individually. Before 
starting the exposure to the virtual scenario, the STAI (S-Anxiety), the SUDS, and the PANAS were 
administered. Afterwards, the participants were immersed in the tropical beach for 7 minutes. Finally, after 
the exposure, the STAI-S, the SUDS, the PANAS, and the IPQ were administered. 


Statistical analysis 

To statistically evaluate the differences in the participants’ scores, we constructed Linear Mixed- 
Effects models (LME models; Bates and DebRoy, 2004) in R (R Core Team, 2013). To obtain p-values 
we used ImerTest R package based on Satterthwaite’s method (Kuznetsova, Brockhoff and Christensen, 
2018). Additionally, to evaluate the relationship between variables, we computed the Pearson’s product- 
moment correlation (Lee Rodgers and Nicewander, 1988). 


Results 


Exposure to natural virtual scenarios can be used to decrease State Anxiety? 

To assess the influence of the Virtual Reality on participants’ anxiety level, the values on State 
Anxiety Level (as measured by State-Trait Anxiety Inventory for adults; Spielberger, 2010) were inspected 
(Figure 3). Given large differences in mean values and standard errors between two groups, a Linear Mixed 
Effect model (LME) with Condition (before and after the VR session) as a fixed effect and Participants as 
a random intercept was constructed to evaluate the differences statistically. The results of the statistical 
analysis provided evidence for the effect of the VR session (for all the estimates see Table 1) meaning 
that the State Anxiety level was significantly decreased by means of VR session (t = -8.55, p <0.001). The 
model results indicate that before the VR session participants on average had a score of 19.48 while after 
the VR session the average score decreased to 12.06. 

Additionally, the values on the Subjective Units of Distress Scale (as measured by SUDS; Wolpe, 
1990) were compared before and after the VR session (for average values see Figure 3). Given large dif- 
ferences observed in average values, a Linear Mixed Effect model (LME) with Condition (before and after 
the VR session) as a fixed effect and Participants as a random intercept was constructed. The results of 
the statistical analysis provided evidence for the effect of the VR session (Table 1) on the values of SUDS 
(t = -7.44, p <0.001). The model results indicate that before the VR session participants on average had 
a score of 28.8 (a score of 30 - “mild anxiety distress”) while after a single VR session the average score 
decreased to 12.16 (a score of 10 - “alert and awake”). Overall, these results provide evidence for the 
effectiveness of the VR session for decreasing state anxiety in the sample of University students. 


State Anxiety Level Subjective Units of Disstress Scale (SUDS) 
20 30 
15 
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Figure 3. The mean values along with the standard errors per participant (Morey, 2008) are 
presented for State Anxiety Level and Subjective Units of Distress before and after the VR session 
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Table 1. 
The results of LME models for effect of the VR session on State Anxiety Level and SUDS 


Model: State_Anxiety_Value~ (before/afterVR)+ (1 | Subject) 


Estimate Std. Error tvalue = — pp value 

(Intercept) 19.48 42 1614 ~~ <0.001™ 

After VR -7.42 0.87 -8.55  <0.001"" 
Model: SUDS_Value~ (before/afterVR)+ (1| Subject) 

— Estimate Std.Error ——stvalue p value 

(Intercept) 28.8 2.19 10.32 — <0.001"* 

After VR -16.64 2.24 -7.44 — <0.001™" 


Can exposure to Virtual Reality be used to decrease Negative Mood and facilitate a Positive 
Mood? 

First, the participants’ basic emotions were assessed with The Positive and Negative Affect Schedule 
questionnaire (PANAS; Watson, Clark and Tellegen, 1988). Based on participants’ responses the mean 
values were calculated along with within-subject standard errors (Figure 4). The visual inspection of both 
positive and negative affect values allowed observing differences before and after the VR session. 

To test these differences statistically, LMEs with Condition (before and after the VR session) as 
a fixed effect and Participants as a random intercept were constructed. The results of the statistical 
analysis provided evidence for the effect of VR session (Table 2) on Negative Affect meaning that the 
level of Negative Affect can be significantly reduced by means of VR session (t = -5.95, p <0.001). The 
model results indicate that before the VR session participants had on average a score of 17.44 while after 
the average score decreased to 12.88. For Positive Affect, participants’ scores increased after the VR 
session, however, no statistical evidence was obtained (t = 1.24, p > 0.05). 


Positive Affect Schedule (PANAS) Negative Affect Schedule (PANAS) 
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Figure 4. The mean values along with the standard errors per participant (Morey, 2008) are 
presented for The PANAS questionnaire before and after the VR session 


Table 2. 
The results of LME models for effect of VR session on Positive and Negative Affect Schedule 


Model: Positive_Affect_Value ~ (before/afterVR)+ (1 | Subject) 


Estimate Std. Error = stvalue =p value ~ 
(Intercept) 32.02 | 1.09 | 29.29 <0.001*** 
After VR 1.1 0.88 1.24 > 0.05 
Model: Negative_Affect_Value ~ (before/afterVR)+ (1 | Subject) 
Estimate —— Std. Error t value pvalue 
(Intercept) 17.44 L 0.81 2141 <0.001*** 
After VR -4.56 0.77 | -5.95 <0.001*** 
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Is there a relationship between State Anxiety Level and the Feeling of Presence? 

The feeling of presence was measured by the IPQ questionnaire (Igroup Presence Questionnaire; 
Schubert, Friedmann and Regenbrecht, 2008). The following values from the IPQ questionnaire were 
analyzed: IPQ total score, the values for the four IPQ subscales: Involvement (INV), General Presence 
(GP), Spatial Presence (SP), Realness (REAL). The mean values for these subscales were visually 
inspected and are presented in Figure 5A. The participants had the highest values for Spatial Presence 
and the lowest for General Presence. 

To assess whether there is a relationship between state anxiety and the feeling of presence, 
Pearson’s product-moment correlation between the values of State Anxiety (as measured by the State- 
Trait Anxiety Inventory for adults; Spielberger, 2010) and the total IPQ score was calculated. A negative 
weak correlation was obtained between State Anxiety Level before VR and IPQ total score, r(48)= - .28, 
p < .05. When comparing State Anxiety Level after VR and IPQ total score similar tendencies were 
observed: r(48)= - .38, p < .01. The visual representation of values for these two variables (Figure 5B) 
indicates a large spread of values and together with the current findings does not provide grounds for 
concluding that there is a correlation between anxiety level and the total score of the feeling of presence. 
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Figure 5. A. The mean values and standard errors are presented for four different IPQ Subscales: 
Involvement (INV), General Presence (GP), Spatial Presence (SP), Realness (REAL). B. State Anxiety 
Level plotted along with IPQ total score before and after the VR session 


To further investigate the connection between the state anxiety and feeling of presence, the 
correlations between four subscales of IPQ and State Anxiety Level before and after VR were computed. 
The correlation values (r) along with p-values are presented in Table 3. When comparing IPQ values 
with State Anxiety scores, significant negative correlations were found between General Presence score 
and State Anxiety before (r(48)= - .43, p < .05) and after VR session (r(48)= - .47, p < .001). Additionally, 
weak significant negative correlations were found between State Anxiety score after VR session and the 
following IPQ subscales: Involvement Score (r(48)= - .38, p < .05) and Spatial Presence score (r(48)= - 
.29, p < .05). Marginal negative correlation was found between STAI score before VR session and Spatial 
Presence score (r(48)= - .27, p = .05). 


Table 3. 
The results of correlation analysis between feeling of presence (IPQ) and State Anxiety Level 
before and after VR 


Model: Positive_Affect_Value ~ (before/afterVR)+ (1 | Subject) 


Estimate = Std. Error t value ___ pvalue 
(Intercept) | ls —_<0.001*** 


Estimate  =—— Std. Error t value p-value 
_ (Intercept) 174400 BT AT S000" 
After VR -4.56 0.77 -5.95 | <0.001*** 


** - correlation significant at level of 0.001, ** - correlation significant at level of 0.05, * - marginally significant 
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Discussion and Conclusion 


In the current study, we aimed to reduce the anxiety level and negative mood and to induce a 
positive mood in healthy individuals that were exposed to a virtual scenario consisting of a tropical beach. 
The study was performed in an individual form. To estimate the impact of VR technologies on the psycho- 
emotional state, the questionnaires were administrated on state anxiety level, subjective units of distress, 
and positive and negative mood. Additionally, the feeling of presence of the participants was assessed. 
The challenge was, therefore, to evaluate whether a single VR session can lead to the reduction of anxiety 
level and negative mood and induction of positive mood. 

To explore the possibility of state anxiety reduction by means of VR, the values for state anxiety 
were compared before and after the VR session. The results of the statistical analysis provided evidence 
regarding the efficacy of VR on the state anxiety reduction reflected in both STAI (S-Anxiety) and SUDS 
values. For STAI (S-Anxiety), before the VR session participants on average had a score of 19.48 while 
after the VR session, an average score decreased to 12.06. It is worth mentioning that both before and 
after the VR session participants were not characterized “high” levels of anxiety or stress (Arnaiz and 
Guillén, 2013; Gevorgyan, Berberyan and Berberyan, 2022; Knight, Waal-Manning and Spears, 1983) 
because of the fact that the sample was composed of a healthy population, namely, a group of university 
students. For SUDS, before the VR session participants on average had a score of 28.8 while after the VR 
session, an average score decreased to 12.16. Overall, these results provide evidence for the effectivity 
of VR sessions for decreasing the levels of anxiety. These results add to the body of the literature on the 
effectivity of VR use for the treatment of anxiety disorders (Diemer et al., 2014; Freeman et al., 2017: 
Gorini and Riva, 2008; Maples-Keller et al., 2017; Wiederhold and Wiederhold, 2005) suggesting that 
VR can be also used to reduce state anxiety in a healthy population. This finding is consistent with the 
previous studies on anxiety reduction in students by means of VR (Camara and Hicks, 2020; Garcia- 
Batista et al., 2022; Harris, Kemmerling and North, 2002). 

To assess whether the feelings and emotions of participants can be manipulated due to a single VR 
session, the resulting values of PANAS questionnaire were compared before and after the VR session. 
The results of the statistical analysis provided evidence for the effect of VR session on the negative affect 
reduction. Thus, before the VR session participants had on average a score of 17.44 while after the VR 
session, an average score decreased to 12.88. For positive affect, participants’ scores increased after the 
VR session, however, no statistical evidence was obtained. We explain this data by the possible lack of 
a necessary balance between positive and negative emotional states: a decrease in negative effects will 
not always automatically lead to an increase in a person’s positive feelings. 

To explore the relationship between the state anxiety and feeling of presence in VR, the Pearson's 
correlation between the resulting values was computed. The results suggest no relationship between the 
total IPQ score and state anxiety level. One of the possible explanations of the current findings is that 
our participants in general were not characterized by a “high” anxiety level that is often associated with 
the feeling of presence (e.g., Regenbrecht, Schubert and Friedmann, 1998). Another explanation of the 
absence of a relationship is that the virtual environments applied in this study were rather relaxing (a 
virtual scenario consisting of a tropical beach) than stressful (Alsina-Jurnet et al., 2011). 

Although VR is not yet extensively used in the clinicians’ daily practice, it has been demonstrated 
to have tremendous potential for the diagnosis and treatment of mental health problems (Freeman et al., 
2017). There is every reason to believe that virtual reality will move from the category of experimental 
diagnostic methods to everyday practice: using VR, you can create real life situations that standardize 
psychological examination, making it more reliable. Despite the skepticism of representatives of the 
“traditional school”, VR will be distributed in psychotherapy, as there are already scientific studies that 
confirm the effectiveness of its use. The question of how exactly VR facilitates the clinical change is still 
open: a discussion is ongoing with the first view that immersion into VR has significantly affected the 
emotional state of the personality and the alternative view that this was rather driven by personal features 
that affect the dependency on it. 

Virtual reality has postulated itself as a new effective technique that can be applied for the study, 
diagnosis, prevention, and treatment of various mental health issues. Additionally, its application has 
recently been extended to reducing negative states in healthy individuals. The current results allow us to 
conclude that VR can be effectively used to improve the mental well-being and quality of live, in general. 
Altogether this would lead to promotion of the harmonious development of the personal as well as its 
potential (Berberyan, 2021). In future, to enable wider application of VR, systematic research should be 
carried on issues including methodology, technical equipment, preparation of participants for entry into the 
virtual environment before the start of the study to exclude mental health risks. This would contribute to 
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the development and implementation of this unique latest technology in the theory and practice. 
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Abstract: The article analyzes the possibility of transitioning a general education school to an online learning format. 
The indicators are such categories as educational activity, quality of education, intensity of education, learning motives. To 
achieve this goal, a sociological survey of high school students in Rostov-on-Don (Russia), as well as their parents and school 
teachers, was conducted. The last two groups of respondents are presented in the status of experts. Based on the analysis of 
empirical data, the following conclusions were made. More than half of all high school students surveyed (66.6%) expressed their 
intention to continue the online learning experience they received during the response to the COVID-19 epidemic. However, the 
conjugations of willingness to study online with the categories “learning motives”, “quality of education”, “intensity of education” 
showed that the high motivation declared by high school students for the learning process does not correspond to their real 
behavior in distance lessons. The main motive for choosing online education for high school students is the convenience of 
this format of education. The survey showed a low degree of significance of other reasons for choosing online education. 
Preferences for online convenience and the desire to learn asynchronously reflect the unmanifested goal of getting out of the 
teacher’s control in order to reduce their educational activity. It can be assumed that this is due to the social immaturity of high 
school students and the lack of understanding by most of them of the value of secondary education. Based on the analyzed 
data, three approximately equal groups of respondents were identified. In the first group, high school students are focused on the 
standard school-lesson system with elements of e-learning (40% of respondents). In the second group, the advantages of online 
learning are articulated, which are associated with convenience and greater resource potential compared to classical learning 
(35% of respondents). The third group represents the interests of high school students, who are not so much interested in the 
format of education as the opportunity to get out of the control of the teacher and find themselves in a convenient educational 
environment to simulate learning activities (25% of respondents). This means that online learning format, the usefulness of 
which is obvious only if students have a stable cognitive activity, is unacceptable for most high school students. 

Keywords: online learning, quality of education, intensity of education, learning motives, high school students, pedagogues, 
parents. 


Introduction 


Online learning has been attracted close interest from scientists for last two decades. It is caused 
by new information and technical possibilities of organizing educational work in a remote format. Modern 
technologies provide instantaneous transmission of educational information over a distance and maintain 
synchronous audiovisual contact between the teacher and the student. Thanks to new technological 
opportunities, a teacher in the classroom is no longer considered by a certain part of society as something 
mandatory for receiving a quality education. This part of society represents online learning as a modern 
educational model that fully meets the requirements and demands of the time. This position is reflected in 
scientific discourse (Golovanova, 2019; Grechushkina, 2021; Smirnova, 2019). However, many scientists 
support an alternative point of view, according to which online learning is seen as a threat to high-quality, 
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intensive educational process (Ivanova and Murugova, 2020; Kuznetsov, 2020; Kovalev and Latsveeva, 
2021). 

It would be wrong to reduce the problem of the relevance of online learning in high school only 
to the technological aspect. The consciousness of children is not a computer program that can be 
filled with all the information necessary for life through technical communication channels. The main 
question is whether Russian schoolchildren have cognitive activity sufficient for the emergence of intrinsic 
motivation for learning activities. Depending on the answer to it, scientists determined the prospects 
for the development of e-learning in high school. In essence, this means that it was through the fact of 
acknowledging or denying motivation that the very possibility of transferring schoolchildren to the digital 
learning format was assessed. When answering this question, all experts are divided into two large 
groups: the first unites scientists who support assertion that schoolchildren are intellectually and socially 
ready to switch to online lessons — in their opinion, the majority of students will not lose motivation (Said, 
2018; Solovieva and Semenova, 2020; Razmacheva, 2021; Kozitsyna, 2021). A different position is held 
by those who do not see high school students as having sufficient cognitive activity to switch to remote 
mode (Markeeva, 2020; Bakaeva, 2016; Nishanbaeva, 2021). The second point of view currently prevails 
in the scientific community. 

The study of the established scientific discourse, however, showed that Russian sociologists had 
not conducted empirical sociological research on the study of the motivation, intensity and quality of 
education of high school students. The present work aims to close this gap in science. 


Materials and Methods 


The theoretical and methodological foundations of the study are based on the approach developed 
in the course of the joint scientific work of V.I. Chuprov and Yu.A. Zubok (Zubok and Chuprov, 2020; 
Chuprov and Zubok, 2008). The authors call it polyparadigm, uniting the most interesting results achieved 
in the previous scientific tradition in study of youth. In the framework of the scientific activities of these 
Russian sociologists, the most significant features that form the sociological definition of youth as an age 
group were integrated. “The variety of these features determines the complex internal structure of youth, 
its differentiation and differences, in which its essential properties are revealed. These are the transition 
of social status, lability, extremeness, transgressiveness of consciousness, increasing globalization and 
new forms of standardization” (Zubok and Chuprov, 2017). These characteristics have been detailed in 
the theoretical works of researchers. In them, young people are characterized by social instability, change 
of interests, mobile shift of value accentuations in the hierarchy of their own needs. The authors argue that 
the personal properties of this age group are extremely variable and this inevitably affects the properties 
of social interactions of young people. It is difficult for young people to fix their interest on one concrete 
thing, they need a change of impressions, a constant feeling of novelty. 

It is necessary to say a few words about the author’s categorical apparatus. It was developed in 
such a way that questions of the questionnaire together constituted a variable image of cognitive activity 
of high school students. The basic concepts differentiated in the questionnaire include: motives, needs, 
personality traits of a high school student, motivations for action, educational goals. 

The empirical base of the study was formed on the basis of a mass sociological survey conducted 
by the authors in January-February 2022. Respondents who received personal experience of learning 
activities in online learning were interviewed. The study involved 860 high school students, 1246 parents 
and 636 teachers living in Rostov-on-Don. To increase the representativeness of the method used, the 
sample included the parents of the surveyed high school students and those teachers who had experience 
of teaching high school students online. Data processing was carried out in the SPSS-22 program. 
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Results and Discussions 


First, we evaluate individual feelings from the learning experience (work, control of high school 
students) during the period of self-isolation and social distancing. We will be interested in attitudes towards 
compulsory online learning. 

Table 1 

Distribution of answers to the question: “What are your individual feelings from the experience of 
distance learning (working at school)?”, % 


Answers 
Set points High school Teaches Pareits 
students 

1.1 would like to continue — it is convenient for 
me 50,0 | 22,9 74 
2.1 would like to continue — it improves the 
_ quality of education 16,6 6,7 3,9 
_ 3.1 am against online — quality of education is 
~ declining 33,8 643 49.3 

4 | am against online — | do not like to study 273 30.8 61.6 


in this format 


The results were partly expected, but quite revealing. More than half of high school students would 
not mind going back to remote mode. But, importantly, the vast majority of respondents who made this 
choice pointed to the convenience of online learning, rather than its ability to provide quality education. 
Meanwhile, for this variable, the respondents could choose two options. However, convenience was 
chosen by 50% of high school students, and only 16.6% expressed confidence that online learning gives 
a quality result. We cannot say that high school students are not interested in quality issues, because 
33.8% chose the option “I am against online - quality of education is declining.” Even more significant 
results were shown by teachers (64.3%) and parents (49.3%), denying the ability of online learning to 
ensure the quality of education. 

The largest scale of rejection of online learning is shown by parents. Among them, only 7.4% chose 
the option “I would like to continue — it is convenient for me” (teachers - 22.9%). 

Supporters of online learning prioritize the convenience of study (work) when choosing this form of 
education. Cautiously for the time being, we will make an assumption that even to the detriment of quality. 
But so far this thesis has no valid evidence. 

In addition to quality, the ability to maintain a high intensity of learning activities in online learning 
should be recognized as significant. Itis understood as the ability to perform acertain amount of educational 
tasks per unit of time, or the ability to engage in educational activities for a certain period of time, or both. 


Table 2 
Distribution of answers to the question: “Did your intensity of learning activity decrease in conditions 
of online learning?”, % 


Answers 
Set points High school Teachers 
students 
1. | (high school student) solved more 
educational tasks, because | find it more 
: ; 32,2 71 
interesting and more convenient to study 
remotely 
2. | (high school student) solved a smaller 
amount of educational tasks due to overwork, 
reduced teacher control and loss of the 30,9 41,9 
opportunity to receive teacher explanations in 
time 
3. It is difficult to answer / It depends on the 36.9 50.9 


individual characteristics of the children 
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There is one important aspect of this variable that needs attention. From the table 1 it follows that 
more than half of high school students are aimed at continuing their studies online. Meanwhile, only 
32.2% of them are convinced that the intensity of their educational activities has increased. It can be 
assumed with some degree of confidence that in this group there is a large stratum of respondents who 
perceive online learning as a way to reduce the intensity of learning activities. 

Connecting with the question of evaluating one’s attitude to online learning will help us understand 
the potential size of the group of high school students for whom the rejection of classroom-contact learning 
can be interpreted as cost minimization. 


Table 3 
Conjugation of answers to the questions: “What are your individual feelings from the experience of 
distance learning (working at school)?” and “Did your intensity of learning activity decrease in conditions 
of online learning?”, % 
_ What are your individual feelings from 
the experience of distance learning 
(working at school) 


2 5 S > 
= a) > om 
i 78 SS of 
E= | ) ~~ 
eo $3 "8 TE 
Did your intensity of learning activity decrease in 2S = 2 2D 2S 
conditions of online learning Se ge ra e2 
2£F of oS os 
= ro oO oe Oc os 
36 38 Ss| 285 
SS $55 &8 6&2 
— _ ii =" = pass 
~8 QELS 3B es 
1. | (high school student) solved more educational tasks, 
because | find it more interesting and more convenient to study 39,3 59,3 49 109 
remotely 
(32,2) _ 
2. | (high school student) solved a smaller amount of 
educational tasks due to overwork, reduced teacher control and 
loss of the opportunity to receive teacher explanations intime 15,0 56,0 46,1 
26,0 | 
(30,9) 
3. It is difficult to answer 347 2,7 391 43,0 
(36,9) | 


Some explanations for the Table 3: in the line record there are three groups in which the assessment 
of the results of the intensity of learning activities in the online learning mode is determined. Frequency 
distributions are given in brackets. The columns indicate the characteristics of attitudes towards their 
own experience of distance learning. The status of the variables has an entry in the columns, because 
we find out how much learning tasks are performed by high school students in each of the four groups of 
schoolchildren who assessed their learning experience in online learning. 

The pairing results show the reliability of the data obtained. Those who negatively assessed their 
experience of online learning, for the most part, are confident in its low intensity. For example, in the group 
“| am against online — quality of education is declining” — 56.0% indicated a decrease in the intensity of 
study (36.1% found it difficult to draw conclusions). There is also a noticeable strong change in relation 
to the average distributions: in the group of high school students who oppose online because of the 
deterioration in the quality of education, the variant “I solved more educational tasks” was chosen by 4,9% 
(for the total sample 32,2%); and by the group of high school students who do not like to study online — 
10,9% (for the total sample 32,2%). 

Other distributions among respondents, where the online experience is rated positively due to its 
convenience. In this group, the attitude to the intensity of study in the conditions of online learning does 
not differ much from the average values for the entire sample. Here, 39.3% of respondents solved more 
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educational tasks (32.2% in the total sample). The increase in the number of high school students who 
successfully solved a larger amount of educational problems, as we see, is insignificant. But this is not 
even more significant. As you can easily see at the intersection of the first column and the second line, 
26% of high school students from this group (online is convenient) chose the answer option “I solved 
a smaller amount of educational tasks due to overwork, reduced control by the teacher and the loss of 
the opportunity to receive teacher explanation in time”. Moreover, the downward differences in the total 
sample are not significant (26%<30.9%). It is in this segment that you need to look for the bulk of high 
school students, for whom online is just a way to reduce the intensity of learning. 

Equally indicative are the results in the second column group: “I would like to continue — it improves 
the quality of education” (16.6%). Deviations from the average frequency indicators are very significant. 
In this group, 59.3% of respondents are convinced that they solve a larger volume of educational tasks 
(32.2% in the total sample). The decrease in the intensity of learning is no longer as noticeable as in the 
group where online learning was chosen for convenience. Only 15% indicated the solution of a smaller 
volume of tasks (30.9% in the total sample). These 15% of high school students are also potentially 
adding to the cohort of those who are aimed at reducing the intensity of their studies. 

Having dealt with the data obtained and revealed their reliability, we then calculate the approximate 
number of high school students for whom the rejection of classical education can be interpreted as a 
conscious decrease in the intensity of educational work. To do this is quite simple. The action will be 
performed in two stages, separately for the groups “online is convenient” and “online is qualitatively”. The 
first group (“online is convenient”) makes up 50% of the entire sample (Table 1). In it, 26% of respondents 
indicated a decrease in the volume of tasks to be solved, and 34.7% found it difficult to answer. Both 
of these figures should be divided by two, highlighting half of the respondents from the total sample. 
Consequently, 13% of all high school students surveyed chose online only for the sake of convenience, 
and 17.3% presumably also, at least in some part of their learning activities. The second group (“online 
is qualitatively”) makes up 16.6% of the entire sample (Table 1). It indicated a decrease in the volume of 
tasks to be solved and found it difficult to answer - 15% and 25.7% of respondents. Relative to the general 
sample - 2.5% and 4.7%. Summing up the results for both groups, it turns out that 15.5% (13% + 2.5%) 
of respondents chose online learning even in the face of a reflexive decrease in the volume of tasks to 
be solved, and 22% (17.3% + 4.7%) conditions of non-reflexible decrease in the volume of tasks to be 
solved. Thus, their choice of the form of education in favor of online is not associated with the success 
of solving educational problems. That is, convenience is a value in itself, acting as a priority in relation to 
intensity. These figures are very symptomatic and indicate a high percentage of respondents (more than 
a third of the number of respondents) for whom intensity in training is not considered a significant choice 
and needs external motivation. 

Returning to the data in the Table 2, we note that more than a third of the surveyed high school 
students (36.9%) found it difficult to determine the state of intensity of their educational activities. The 
reasons for this may be different, but it is important for us to understand that more than a third of the 
respondents demonstrate that they do not have the proper indicators for evaluating the effectiveness of 
online learning. This is important from the point of view of learning motivation, because in this selected 
group, motives lose their connection with the results of education. This group can be defined as not being 
indifferent to learning results, but not having the ability to objectively assess the intensity of their learning 
activities. 

The teachers were extremely critical. Only 7.1% supported the position according to which the 
intensity of learning activity in the context of online learning is growing (the fact of a decrease was noted 
by 41.9%). It is hardly possible to question their expertise on this issue. Let us assume that teachers 
themselves work more effectively in the online format, and this is inevitably reflected in the growth of the 
intensity of involvement of schoolchildren in the educational process. At the same time, more than half 
of teachers (50.9%) felt that the intensity of online learning “depends on the individual characteristics of 
children.” Such a high percentage of answers for this position suggests that the effectiveness of learning 
activities for some students in online learning is increasing. 

We did not find out the point of view of the parents, because considered that they did not have 
expert competence on this issue. 

Next, we will connect the issues of quality and intensity of educational activity with motivation itself. 
To begin with, we present the results of the simplest (frequent) measurement of the state of motivation in 
online learning. 
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Table 4 
Distribution of answers to the question: “Have you noticed in yourself (in high school students) a 
decrease in motivation for learning when switching to distance learning?”, % 


Answers 
Set points 
High school Teachers 
students 

1. No, homeschooling is convenient 46,4 56 

2. No, learning online is very productive / No, 

high school students in any form of education 10,8 11,4 
understand well why they need knowledge 

3. In part, studying at home is difficult / In part, 327 436 


some definitely pretend to study 

4 Yes, learning at home is almost impossible / 

Yes, learning at home leads to partly imitation 10,1 39,3 
of learning due to sharp decrease of quality 


It is easily to see that the positions of high school students and teachers diverge quite strongly. 
The most important difference stems from the assessment of the relationship between convenience and 
motivation. 46.4% of the surveyed high school students are convinced of the growth or maintenance 
of the previous level of motivation, because “homeschooling is convenient”. Only 5.6% of the teachers 
surveyed agreed with this judgment. This range of opinions can partly be explained by the fact that the 
students spoke about themselves, and the teachers gave an expert assessment. And the answer “some 
definitely pretend to study” looked much more attractive. Understanding the risks of underrepresentation, 
we added the ability to select two set points to the teacher question variable. However, this did not 
increase the popularity of choosing the first two options. Teachers did not see a significant relationship 
between convenience and increasing motivation. 

Equally diametrically different is the position between the two groups of respondents and in the 
opposite position, according to which a decrease in motivation is claimed in the conditions of online 
learning (fourth line of the table 4). Only 10.1% of the surveyed high school students considered “learning 
at home is almost impossible”. Among teachers, such categoricalness was supported by 39.3% of 
respondents who chose the option “learning at home leads to partly imitation of learning due to a sharp 
decrease in quality”. 

If evaluated by the aggregate, the least likely option was “online is very productive” (10.8%) / “high 
school students in any form of education understand well why they need knowledge” (11.4%). High school 
students and teachers were given markedly different definitions. Pupils were asked about the quality of 
online learning, teachers about the presence of certain personality traits in high school students, with 
which society associates the onset of social maturity. In fact, teachers spoke about the lack of social 
maturity among high school students, and schoolchildren ignored the category of “learning productivity” 
as significant for themselves. 

From this position, let's turn to a meaningful description of the relationship between motivation and 
the assessment of the intensity of learning activity in online learning. 

We have already measured the linear distribution with respect to the intensity of online learning 
(Table 2). Recall that among high school students, three groups of approximately the same size were 
distinguished, in which one part of the respondents (32.2%) showed confidence in the growth of intensity 
in online learning, the second indicated its decrease (30.9%), and the third found it difficult in the choice 
of answer (36.9%). Motivation was measured separately from intensity, and slightly more than half of the 
total number of students surveyed noted that it did not decrease online (Table 4). 

Now we need to use the “intensity” criterion to characterize the degree of involvement in educational 
activities of both motivated and demotivated high school students. This will help test the validity of 
judgments about their high motivation in online learning environments. This must be done for two reasons. 
First, any judgments of schoolchildren should be subjected to critical evaluation. Secondly, the opinions 
of high school students and teachers are diametrically opposed. As a test criterion, the questionnaire 
included the question “How often did you do extraneous activities during distance lessons?”. Variants of 
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the proposed answers for this criterion acts as given values. The answers to the question about motivation 
are in the status of the variables that are being measured. 


Table 5 

Conjugation of answers to the questions: “Have you noticed a decrease in motivation for learning 
when switching to distance learning?” and “How often did you do extraneous activities during distance 
lessons?”, % 


Have you noticed a decrease in motivation for 
learning when switching to distance learning? 


> Q a 
© - D 
n a 5 a 
How often did you do extraneous activities during =o a = E 
distance lessons? = = = a 
E 5 6£ 8 
oS Le) no} Cc 
6) < = ‘= 
Er = © So 
S ® os £ 2s 
sic o8eq ®3e 88s 
zt eo} a c- GO a 
=8€ N8S aSS TEE 
1. | was not distracted at all, | was constantly included 
in the educational process 35,8 31,5 141 12,9 
(30,7) a : 
2. It is hard to keep attention, but | tried 
33,1 43,5 43,1 25,9 
(36,6) 
3. About half the time | switched to my own affairs 
j 18,4 163. 27,9 24,7 
(19,6) 
é a of the time minding my own business 91 76 127 153 
5. Included in the lesson only when my teacher 
addressed me personally 3.6 1,1 2,2 21,2 
(4,6) 


In Table 5 we have four groups of respondents. The first consists of those who find it convenient to 
study online; the second unites those who chose the option “learning online is very productive”; the third is 
formed from respondents who considered that “studying at home is difficult’; the fourth was formed through 
the choice of the judgment “learning at home is almost impossible”. The first two groups can be considered 
motivated in terms of online learning, the third is partially motivated and the fourth is demotivated. The 
data is contained in four columns. The given values for them are judgments that characterize the intensity 
of the educational activity of the degree of readiness of schoolchildren to be included in the educational 
process online. The purpose of this conjugation model is to check the reliability of the answers of high 
school students in Table 4, which shows that 57.2% (46.4% + 10.8%) of respondents (in the first two 
groups of motivated ones) stated that their motivation in online learning does not decrease. 

As a hypothesis, it would be reasonable to assume that the intensity of education in the two 
motivated groups will be noticeably higher than in the partially motivated or unmotivated group. In addition, 
two groups of motivated schoolchildren should be expected, if not a 100% choice of the given value “I was 
not distracted at all, | was constantly included in the educational process”, then at least within the limits 
of 60%-70%. However, this hypothesis was only partially confirmed. Yes, indeed, in the two motivated 
(in terms of self-presentation) groups, a significantly smaller part of the respondents were distracted from 
the educational process (the first line of Table 5). But this is not even half of the number of the first and 
second groups. In the first group, 33.1% chose the option “It is hard to keep attention, but | tried”, and in 
the second group even more — 43.5%. 18.4% in the first group and 16.3% in the second “About half the 
time | switched to my own affairs”. 9.1% and 7.6% and even “Most of the time minding my own business’. 

Let us calculate, according to the same scheme, according to which the data in Table 3 were 
calculated, the percentage of high school students from the total sample, who spent half or more of 
their time doing extraneous activities during distance lessons. There are 15% of those in the two groups 
of motivated people. To these can be added 20.2% of those who had difficulty concentrating in online 
classes. We remind that these percentages are not within the group of motivated high school students, 
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but from the general sample. Intragroup distributions can be seen in the Table 5. 

These figures show that the high level of motivation among high school students who stated that 
they have no problems with motivation in the online learning format does not correspond to the real state 
of affairs. Obviously, this means that in the structure of the personality of high school students there are 
social qualities that do not contribute to maintaining motivation at a high level in the process of online 
learning. 

We use one more criterion to test the subjectively presented ability to find intrinsic motivation in the 
conditions of online learning: “What online learning format do you prefer to study in?”. Respondents were 
asked to choose three set points: asynchronous, synchronous, and regular school. Objectively, only highly 
motivated people can learn effectively in asynchronous mode, because in it a priori there are no external 
impulses to motivation. About synchronous online and standard school education, it is unreasonable to 
make unambiguous conclusions about motivation on the proposed grounds. Therefore, the unconditional 
willingness to go to a distance can be seen only among motivated supporters of online learning. 


Table 6 
Conjugation of answers to the questions: “Have you noticed a decrease in motivation for learning 
when switching to distance learning?” and “What online learning format do you prefer to study in?”, % 
Have you noticed a decrease in motivation | 
for learning when switching to distance 


learning? 
> = 
® a2) 
; 2 2 5 @ 
What online learning format do you prefer to study in? D> ‘oO = 5 
i & wz <= 
8 s 2 ® 
<= > Lo) 
is e 3 = 
EE = 2 G ® 
So ® os = =| 
[oe -2 Ss Boe, eee 
Sw col om Lox 
=8¢ 582 sS8 cFe 
1. Asynchronous: without teacher, completely self-guided 
(22,8) 7,2 19.5 90 71 
2. Synchronous: when a teacher at computer conducts a 
lesson as if in a classroom 39,7 37,0 151 59 
3. It is better to study in a regular class at school 
(49,2) 231 435 76,0 87,1 


Presented in the Table 6 data give us the opportunity to clarify two positions. First, to find out 
if all high school students who find the online format convenient or productive want to abandon the 
standard school education. Secondly, does the awareness of the fact of a decrease in motivation in the 
conditions of online learning lead to a negative attitude towards the synchronous or asynchronous format 
of educational work. 

We are going to answer the first question first. Remind that the group of high school students, in 
which motivation does not decrease in the conditions of online learning, is divided into two subgroups: 
in one group, motivation does not decrease, because online is recognized as convenient; in the second 
because of its productivity. It should be recognized that in the first subgroup (online is convenient), 
according to the results of conjugation, 23.1% chose the answer option “It is better to study in a regular 
class at school’, and in the second subgroup (online is productive) 43.5%, that is almost half. Such 
distributions at least show that the intention to continue the online learning experience, which was stated 
by 66.6% of the surveyed high school students (Table 1), does not at all mean their abandonment of the 
classroom system. This aspect of the analysis gives us additional information to conclude that the first 
group of motivated is extremely heterogeneous. This was already noticeable from the data included in 
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tables 4 and 5. In this group, there is a fairly high percentage of those who are indifferent to both the 
intensity and quality of training. Some details on these aspects are given to us by pairing the question 
of motivation with the question of preferences for the format of training. Thus, an indicative downward 
trend is visible within the “motivated to online learning” group: of those who associate motivation with 
convenience, only 23.1% choose the intention to study at a regular school; where staying motivated 
Online is associated with the “productivity” category, 43.5% of high school students would prefer to study 
in a regular school. We also consider it no coincidence that asynchronous learning is most often chosen 
by those schoolchildren who are primarily focused on the convenience of the online mode (37.2%). Those 
who have not lost motivation in the online format, but who associate it not with convenience, but with the 
growth of their own learning productivity, choose asynchronous learning much less often (19.5%). It can be 
assumed that such significant preferences for online convenience and the desire to learn asynchronously, 
in fact, reflect the goal that is not manifested openly to get out of the control of the teacher in order to 
reduce their educational activity. 

With regard to the group where motivation in the conditions of online learning is lost or partially lost 
(the third and fourth columns in Table 6), everything is very clear. Those who have partially or completely 
lost motivation are aimed at studying in a regular class (subgroup “studying at home is difficult” - 76.0% 
and subgroup “it is better to study in a regular class at school” - 87.7%). However, in these two subgroups 
there is a small part of high school students who, even with motivation lost online, choose synchronous 
or asynchronous learning. Our calculations showed that it is 9.1% of the total sample. With a high degree 
of probability, we can assume that these are schoolchildren who are not interested in either the quality or 
the intensity of learning activities. 

A significant result of the survey was the identification of three groups of high school students: 1) 
reducing the performance of online learning and opposing this format; 2) reducing the rates of online 
learning and advocating this format; 3) supporters of this format that increase the performance of online 
learning. In terms of size, the groups are approximately equal with some preponderance towards the first 
of the three. But these are schoolchildren’s ideas. Let us check them through the expert judgments of 
teachers and parents. 


Table 7 
Distribution of answers to the question: “Are high school students divided into those who are better 
off studying remotely and those who are better off studying in the classroom”, % 


Set points Answers of teachers 


1. All children show the best results in online learning 5,9 


2. Some high school students are better off studying online: 


homebodies, disabled, shy, highly motivated “0 


3. Transition to online had a negative impact on the quality of 51.0 
education for all high school students 

Teachers, in general, confirmed the presence of the three groups listed above, but the quantitative 
distributions turned out to be quite different. Thus, only 5.9% of the surveyed pedagogues indicated that 
“all children show the best results in online learning”. At the same time, 43.1% of teachers confirmed that 
among high school students there are children who are disabled, homebodies, shy, highly motivated, who 
are better off learning in electronic format. And finally, the majority (51%) agreed with the opinion that 
“transition to online had a negative impact on the quality of education for all high school students”. 

The opinions of teachers are also confirmed by the consolidated point of view of parents. 
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Table 8 
Distribution of answers to the question: “When the school worked remotely, did you have conflicts 
with your child because of study issues?”, % 


Set points Answers of parents 
1. No, he/she did the right thing 39,7 
2. No, | have no control over his/her studies. 131 


3. Yes, | saw how during the lesson he/she was engaged in 17.8 
extraneous matters , 
4. Yes, | had to force him/her to study 29,4 


Almost half of the parents indicated that they had conflicts with their children caused by an 
insufficient level of motivation to study online. In fact, there could be more of them, but we must take into 
account that not all parents have opportunity to control their children due to professional employment. The 
percentage received is only little more than what the high school students themselves said, referring to the 
decrease in their level of motivation during distance learning. 

And finally, the last one. In the emerging discourse, many scientists, teachers and parents share the 
opinion that online learning is a modern model of education. Some suggest transferring school education 
to electronic format right now. Such projects were actively discussed during the pandemic in the context 
of the initiatives of the head of Sberbank, Herman Gref (School, 2019). After the end of the pandemic, 
the Internet was filled with advertising messages of various kinds of online schools that operate as an 
alternative to the regular high school. The main focus of advertising content unfolds through a description 
of the advantages of online education compared to studying in a regular school. This advertisement and 
the very functioning of the institution of alternative online learning became possible due to the presence in 
the Law on Education of Russia of a legal norm that gives parents the right to transfer their child to family 
education (clause 3, part 1, article 17). This construction is used by entrepreneurs in the educational 
services market to advertise their activities, without bearing any responsibility for the quality of education, 
because. Responsibility shifts entirely to the child’s parents. In this regard, we decided to update and 
personalize the problem by formulating the variable in such a way that the respondents (teachers and 
parents) answered the question about their readiness to transfer not some abstract (foreign) children, but 
their own children to online learning. 


Table 9 
Distribution of answers to the question: “Would you choose online education for your child instead 
of acommon school?”, % 


Answers 
Set points 
Parents Teachers 

1. Yes, in terms of online learning high school 54 34 
students develop intrinsic motivation better , , 
2. Yes, modern high school students study 

56 8.6 
better at the computer 
3. No, high school students drastically reduce 
the intensity of education, when switching to 41,9 412 
online 
4. No, online learning outcomes are worse than 
in the classroom system 52,3 49 6 
5. | would choose it as additional education 32.5 39,8 


The results obtained show that our experts do not accept online learning uncompromisingly. The 
percentage of teachers and parents who are ready to choose online education for their high school 
students instead of a general education school turned out to be negligible. The experts considered that in 
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the context of distance learning, schoolchildren do not develop intrinsic motivation at the required level, 
and the intensity of education and results of educational activities themselves do not meet the established 
standards. More or less significant interest in online learning is associated only with its implementation 
in the status of additional education. The data obtained could have been even worse for the supporters 
of online education, if some of the answers had not been pulled over by the option “I would choose it as 
additional education”, because respondents were given the opportunity to make only two choices of given 
values. 


Conclusions 


The obtained results can be structured according to three main positions: the attitude to online 
learning, the perception of one’s own motivation in the conditions of online learning, and the expert 
evaluation of the online learning work of high school students by teachers and parents. 

Attitude towards online learning. The survey showed that the majority of schoolchildren (66.6%) 
are set to repeat the experience of online learning. However, out of this number, 50.0% of respondents 
perceive online as a form of education that is convenient for them, and only 16.6% associate it with 
an opportunity to improve the quality of education. As for the ability of the online format to support the 
intensity of learning activities, in the process of analyzing empirical data, three approximately equivalent 
groups were identified: 32.2% solved more learning tasks, 30.9% of respondents managed to complete 
a smaller volume, and 36.9% found it difficult to answer. The presence of a large number of set values 
who found it difficult to choose indicates the inability of this part of high school students to realize their 
own intensity of educational work in online learning, which casts doubt on the educational value of their 
choice of distance format. This is especially true in the context of the fact that in this group (36.9% of 
those who found it difficult to answer), 46.6% expressed their intention to continue the online learning 
experience. The fact that it is not worth taking literally the desire of high school students to study online 
without a critical reassessment is also evidenced by the fact that about a third of the students in the group 
who positively assessed the experience of distance learning (26% - “online is convenient’; 15% - “online 
provides quality”) decided less the volume of educational tasks, which indicates a focus on imitation of 
education. 

Perception of one’s own motivation in the context of online learning. A direct question to high school 
students about the state of their motivation made it possible to single out four groups of respondents: 
motivation does not decrease, because learning online is convenient (46.4%); motivation does not 
decrease, because study online productively (10.8%); motivation is partially reduced, because studying 
at home is difficult (32.7%); motivation is reduced, tk. studying at home is almost impossible (10.1%). 
As you can see, 57.2% (46.4% + 10.8%) of high school students did not notice a decrease in motivation 
when switching to online learning. However, their chosen setpoints once again showed convenience over 
productivity. Convenience, of course, is a significant condition for increasing motivation in the educational 
process, but it can hardly be called decisive. And in this case, it can even act as a factor that reduces the 
readiness of high school students for intensive and high-quality education. This hypothesis was tested 
with two questions. 

The first question is about the frequency of doing extraneous activities during distance lessons. In 
the group of those motivated because of the convenience of online learning, only 35.8% of respondents 
were not distracted by extraneous matters, and in the group of those motivated because of online 
opportunities to provide productive learning - 31.5%. The conducted pairing showed a low degree of 
reliability of the answers of high school students that their motivation does not decrease in the conditions 
of online learning. 

The second question is about the preferences of the learning format: asynchronous, synchronous 
and regular class. This is a criterion in order to identify the relationship between the presence of motivation 
in an online environment and the willingness to abandon the traditional class-lesson system. It was found 
that in the group motivated because of online convenience, 23.1% want to study in a regular school, and 
in the group motivated because of online productivity - 43.5%. This means that even the presence of 
motivation in online lessons is not associated with greater productivity of digital education compared to a 
traditional school. And, conversely, some high school students from the two groups of partially motivated 
and demotivated online choose this particular format of learning. According to our calculations, they make 
up 9% of the total sample. Their aim is obvious: to get out of the control of the teacher. 

All this confirms the hypothesis that the high motivation declared by high school students for the 
online learning process is actually more of an intention which is difficult to implement than a reality. It can be 
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assumed that such significant preferences for online convenience and the desire to learn asynchronously, 
in fact, reflect the goal that is not manifested openly to get out of the control of the teacher in order to 
reduce their educational activity. Obviously, this means that in the structure of the personality of high 
school students there are social qualities that do not contribute to maintaining motivation at a high level 
in the process of online learning. 

Expert evaluation of educational work of high school students in online format by teachers and 
parents. Teachers (74.7%) and parents (81.8%) reacted negatively to the experience of high school in 
Online learning, explaining this by the decline in the quality of education and the inconvenience of this 
format. Only 7.1% of teachers are convinced that high school students solved more learning problems in 
online lessons. 43.6% of pedagogues have encountered an attempt to simulate the learning process by 
students, and 39.3% believe that online learning is a complete imitation of the educational process. 51.0% 
of teachers believe that the transition to online has had a negative impact on the quality of education for all 
high school students. However, 43.1% specify that it is more convenient for some categories of children 
(homebodies, disabled, shy, highly motivated) to study remotely. 47.2% of parents came into conflict with 
their children due to the fact that they were engaged in extraneous activities during the lesson. And finally, 
only 5.6% of parents and 8.8% of teachers would like to transfer their children to online education. The 
latest data is especially characteristic of pedagogues: 25.3% of them would like to repeat the experience 
of online learning at school, but only 8.8% choose it for their own children. This confirms the point of view 
of some parents that a number of teachers advocate online learning only for their own convenience. 

Based on the analyzed data, three approximately equal groups of respondents can be distinguished. 
In the first group, high school students are focused on the standard school-lesson system with elements of 
e-learning. According to our calculations, it consists of 40% of respondents. The second group articulates 
the advantages of online learning, which are associated with convenience and greater resource potential 
compared to classical learning. It contains about 35% of the respondents. The third group presents the 
interests of high school students, for whom, from the point of view of solving educational problems, it is 
not so much the format of education that is important, but the opportunity to get out of the control of the 
teacher and find themselves in an educational environment that is convenient for themselves. This group 
is formed from 25% of the respondents. The error in calculating the number of groups can be no more 
than 3-5%. 

The main conclusion: in the presence of a dichotomy between quality and convenience, high school 
students choose convenience non-reflexively. This means that the online learning format, the usefulness 
of which is obvious only if students have a stable cognitive activity, is unacceptable for most high school 
students. 
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Abstract: The knowledge that is based on science, technology, engineering, and mathematics is the basis for the 
development of any country. Less developed countries lack experts in these areas. Therefore, the ENABLE-BIH project (Enhancing 
and Advancing Basic Learning and Education in Bosnia and Herzegovina) was introduced in Bosnia and Herzegovina, which 
aims to improve the situation in the education sector. This study included the Public Institution “Ninth Elementary School” in 
Brcko District of Bosnia and Herzegovina in which this project was implemented. The study included a total of 125 students 
from this school. The aim of this study is to examine the difference between attitudes about STEM from the point of view of 
gender differences and the age of students. After the data were collected, the statements were grouped into appropriate factors 
using factor analysis. The factor analysis showed that five factors stand out in this research. The results of multiple regression 
analysis showed that there is no difference between students ‘attitudes regarding gender differences, while there is a difference 
regarding students’ age. The results of this research showed that the ENABLE-BIH project delivered good results and suggests 
the importance of implementing similar projects in the future. 

Keywords: STEM, Brcko District of Bosnia and Herzegovina, ENABLE-BIH, multiple regression analysis, factor analysis. 


Introduction 


The country’s economic growth and development should be based on new technologies and 
innovations (James, 2021). Science and engineering are the basis for economic growth and development 
(Sahin-Topalcengiz and Yildirim, 2019). Nowadays, more investments are done in educating staff who 
will apply this new knowledge based on science, technology, engineering, and mathematics (STEM). 
The best example of the former is the field of energy efficiency (Radojevic, Nikoli¢ and Skerlic, 2020). 
The best example of the former is the field of energy efficiency (PaSali¢ et al., 2020). However, many 
countries have challenges with STEM education (Beswick and Fraser, 2019). Tran (2018) pointed out that 
innovation is crucial for the development of the United States of America (USA), but it lags behind other 
developed countries in the production of its own qualified staff with STEM competencies. The situation is 
similar in other countries where demand for STEM skills is growing (Marginson, et al., 2013). Developing 
countries are increasingly investing in STEM education (Kelley and Knowles, 2016), and this trend has 
been present in Bosnia and Herzegovina (BiH) in recent years. The biggest challenge in the education 
of staff for STEM areas is the lack of interest of students to study these areas (Waite and and McDonald, 
2019). Therefore, it is necessary to gradually introduce these areas to students in the earliest educational 
process (Ghanbari, et al., 2023). 

Bosnia and Herzegovina has begun the introduction of STEM areas in teaching in secondary and 
primary schools in recent years. The ENABLE-BiH project (Enhancing and Advancing Basic Learning 
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and Education in Bosnia and Herzegovina) was introduced in 2016 in Bosnia and Herzegovina, with 
the aim to improve learning about STEM courses. The basic idea of this project is to avoid focusing 
on special or separate disciplines and connecting these courses through practical application. With this 
approach, students can acquire the key skills and competencies necessary for further progress in the 
modern environment. This project was implemented throughout the whole country, and 12 schools and 4 
areas in Bosnia and Herzegovina were selected. Two schools from the area of the Brcko District of Bosnia 
and Herzegovina (BDBiH) have participated in this project. BDBiH is a unique administrative-territorial 
unit of local self-government under the sovereignty of Bosnia and Herzegovina, which represents its 
specificity in relation to other areas in the country. Thus, this is the reason for special research interest in 
the implementation of this project in BDBiH. 

The aim of this study is to examine the attitudes of students towards STEM teaching implemented 
within the ENABLE-BiH project, which is implemented in the Public Institution “Ninth Elementary School” 
in BDBiH. The special research attention is focused on differences in attitudes towards STEM teaching 
within the ENABLE-BiH project in relation to the gender differences of students and students who attend 
general and specialized classes. The contribution of this research is reflected in the following: 

- Obtaining significant information from students related to STEM classes; 

- Determining the existence of differences between students’ attitudes; 

- Providing guidelines for the improvement of STEM teaching in primary education; 

In addition to the introduction, this paper consists of five other sections. The second section provides 
the theoretical framework for the study. The third section oversees the research methodology. This section 
explains how the research was conducted, presents a research sample, and set research hypotheses. 
The fourth section provides the findings of the research. In this section, the collected claims are grouped 
into factors, and then the hypotheses are tested through multiple regression analysis. In the fifth section, 
the obtained results are explained in more detail through discussion, and the findings are compared with 
other similar studies. The sixth section presents the most important research results providing guidelines 
for future research. 


Theoretical framework 

The concept of STEM education is not a new concept as some authors claim, but this term 
appeared in the 19th century at Harvard University as part of the standardization of the agricultural school 
system (Ostler, 2012). After that process, this term was not used often in education, only to come to 
attention in the 1990s when the National Science Foundation in the United States identified areas crucial 
for improving economic work in the United States (Herro, et al., 2017). The acronym STEM stands for the 
education of students in the fields of natural sciences, computer science, engineering, and mathematics. 
STEM is a specific educational program, designed with the aim of using knowledge from the STEM fields 
when solving problems in everyday life (Rizvanovié and Alihodzi¢, 2019). 

The term STEM refers to the integration of science (S), technology (T), engineering (E), and 
mathematics (M) which relates to both workforce and everyday life experiences (Pimthong and 
Williams, 2018). The STEM term is relevant because every scientific discipline encounters complex 
and multidimensional problems that need to be solved. To make full use of STEM potential in students, 
schools need to improve STEM education and improve teaching (Margot and Kettler, 2019). Gomez 
and Albrecht (2013) advocate the establishment of STEM education and teaching in schools through an 
interdisciplinary approach. Establishing STEM education in school is an innovative approach to education, 
it is widely applied in the global environment and is the basis for the development of educational policies 
and reforms (Nguyen, Nguyen and Tran, 2020). The principle of introducing STEM teaching in schools 
emphasizes the importance of students’ understanding of science and mathematics but all to be integrated 
through theology and engineering (Chesky and Wolfmeyer, 2015). The integration and application of 
STEM concepts and processes are needed by students who should have the opportunity to participate 
in real multidisciplinary situations (Pimthong and Williams, 2018). The application of STEM teaching with 
real examples opens up perspectives for students to learn the basics of using STEM disciplines. 

The importance of students’ education in STEM areas has gained importance worldwide, and 
demand for these skills is growing. However, students drop out from these courses, and almost 50% of 
students do not complete these courses (Chen, 2013). The situation is similar in Germany (Heublein, et al., 
2012) and in other countries. In the last twenty years, a decrease in the number of students who choose to 
continue their education in STEM fields has been observed in many European countries (Babarovic, Pale 
and BuruSi¢, 2018). It has been proven that lower achievements in STEM courses in primary education 
can influence the choice of continuing education in STEM areas (Wang and Eccles, 2013). To solve 
this problem, an adequate way to increase students’ interest in STEM subjects is sought (Sadler, et al., 
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2012). Rosenzweig and Wigfield (2016) point out that STEM educational programs conducted in the 
upper grades of elementary schools and high schools motivate students for STEM areas and increase 
their aspirations toward STEM careers. That is the justification why there are more projects in the world 
that are being introduced in primary and secondary schools in order to increase students’ interest in 
studying in these areas. The introduction of STEM in education gives high autonomy and independence 
to students in solving problems in STEM, creates content and activities tailored to the individual needs 
of students, and there is an interactive approach in teaching aimed at developing student competencies 
(Rizvanovié and Alihodzi¢, 2019). 

Based on the recommendations of the European Commission for Education, which focused on the 
promotion of key competencies and their inclusion in the education system, the Agency for Preschool, 
Primary and Secondary Education (APOSO) in 2012 identified key competencies to be developed in 
Bosnia and Herzegovina. In 2016, following these recommendations, the ENABLE-BiH project was 
launched in Bosnia and Herzegovina, which aims to strengthen education in STEM areas. The goal of 
this project has been to contribute to the process of changing the educational paradigm in Bosnia and 
Herzegovina to enable the development of society and a knowledge-based economy (Mrdovic, 2018). This 
project seeks to set the framework and create a comprehensive basis for the introduction of STEM areas 
in regular education within existing and / or innovative teaching content. Furthermore, this project aims to 
integrate different disciplines such as natural science, mathematics, and technology through STEM into a 
cohesive learning paradigm based on practical applications (Rizvanovié and Alihodzi¢, 2019). Following 
these interests, a common core curriculum was formed in Bosnia and Herzegovina, composed by the 
Agency for Preschool, Primary, and Secondary Education (APOSO). In addition, operational curricula for 
STEM competencies have been designed in order to align the results in mathematics, physics, chemistry, 
biology, geography and informatics, as well as for the courses environment, nature and society, and 
similar courses with global trends and standards. Therefore, it is important to investigate how students 
perceive teaching in STEM areas and what their attitudes are towards STEM. 


Materials and Method 


In this section, an overview of the survey questionnaire, the research and the basic characteristics 
of students who took part in the survey are presented. Additionally, the research hypotheses based on 
previously conducted research are introduced. When conducting the research, primary research will be 
used using questionnaires. The collected questionnaires are coded and prepared for analysis using the 
SPSS (Statistical Package for the Social Sciences) 20 software package. The following analyzes will be 
calculated with this software package: exploratory factor analysis, correlation analysis and regression 
analysis. 


Survey data 

The ENABLE-BiH project was implemented in 12 schools in different regions of Bosnia and 
Herzegovina. Seven primary schools and five secondary schools were included. A total of 2556 students 
were included in this project, of which 1636 were students from primary schools, while 920 students were 
included from secondary schools. The aim of this project was to examine the differences in attitudes 
towards STEM studying in relation to gender differences of students and in relation to the differences in 
form of classes looking at general courses and specialized courses, the basic research sample included 
students of primary schools who took a part in the project. Due to technical and other issues encountered 
in this study and due to the specifics of Brcko District of Bosnia and Herzegovina, this research was 
conducted in Public Institution “Ninth Primary School” Mao¢a. The study included students of the second 
and third triad who were sent survey questionnaires. A total of 125 students from this school were included, 
which represents 7.64% of the research population from this school. 

For the purposes of this study, the research was conducted in the Public Institution “Ninth Elementary 
School” located in Brcko District of Bosnia and Herzegovina. This research included students of the 
second and third triad who were asked to take a part in the survey. The survey questionnaire consisted 
of two parts. The first part of the questionnaire referred to the characteristics of students in terms of a 
gender, form of classes and the success of students in the previous period (Figure 1). The second part of 
the questionnaire consisted of statements related to students ‘attitudes about students being required to 
give their grades ranging from 1 to 5, where 1 was the lowest grade and 5 was the highest grade. A total 
of 21 questions were used, which are presented in Table 2. 

Data was collected from students of the selected school in two ways. Survey questionnaires in 
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paper form were distributed to general class students, while electronic survey questionnaires were sent 
to students of specialized courses. The justification of this approach one can find in the fact that general 
class students do not have the informatics course in their curriculum, while specialized class students 
have this course included, thus these questionnaires were sent as part of the course activity. 


Gender Form of classes 


66; 53% 59; 47% 58; 46% 


= Male 7; 54% 
mu Female 


™ General 


m Specialized 


Figure 1. Characteristic of students 


A total of 149 questionnaires were collected. However, only 125 questionnaires were included in 
the analysis because the rate of their completeness rose more than 80%. Survey questionnaires that 
were completed below 80% were not included in the survey, thus 24 survey questionnaires were excluded 
from further analysis. All survey questionnaires were coded and included in the statistical package SPSS 
(Statistical Package for the Social Sciences). Using this software, students’ attitudes were first grouped by 
exploratory factor analysis (EFA), followed by the examination of the reliability of the measurement scale 
using Cronbach’s alpha (CA) indicators, and then the research hypotheses were examined using multiple 
regression analysis (VRA). 


Research hypotheses 

The study of STEM areas is influenced by many factors. Babarovi¢, Pale and Burusi¢, (2018) in 
their research highlighted the following factors: gender differences, student age, and student motivation. 
Reinking and Martin, (2018) pointed out that gender factors are key to accepting STEM study, and at 
the same time they influence students attitudes. In some other studies, it is pointed out that gender 
differences are decisive factors for acceptance of the STEM areas (Mavriplis, et al., 2010; Robinson and 
Lubienski, 2011; Legewie and DiPrete, 2014). Therefore, this research is focused to investigate gender 
differences and the age of students on their attitudes about STEM study. 

By studying gender differences, no unambiguous results were found by Rosenzweig and Wigfield 
(2016) who concluded that gender does not moderate the systemic effect of motivational interventions. 
From the above, it can be expected that future STEM interventions will be equally effective for all students. 
However, the findings by Huppatz and Goodwin (2013) showed that girls lose the most interest in STEM 
areas between the ages of nine and twelve. Furthermore, Kuschel, et al. (2020) showed that there are 
a small number of women entrepreneurs in STEM areas, which emphasizes the existing gender biases 
and systemic shortcomings in social structures. However, the results showed that since the beginning of 
the 1990s, the number of women who have graduated or obtained their master’s or doctoral degrees in 
STEM areas has increased (Reinking and Martin, 2018). Still, women are less represented in STEM areas 
(Legewie and DiPrete, 2014), which is confirmed by research conducted by the authors Wang and Degol 
(2016). Based on this, the following hypothesis is set for this study: 

H1 - Gender differences in students affect the attitudes toward teaching within the framework 
ENABLE-BiH project 

Although there is not enough evidence to draw firm conclusions that intervention programs are 
effective for younger students, there is general knowledge that supports this assumption (Babarovic, Pale 
and BuruSi¢, 2018). It has been also shown that motivation and interest in STEM areas decline during the 
senior years of primary school (Bryan, Glynn and Kittleson, 2011). This is especially present with the female 
population because they lose interest in the STEM areas during studies, and this is especially present for 
mathematics courses (Robinson and Lubienski, 2011). Daugherty, Carter and Swagerty, (2014) suggested 
that in order to increase students’ interest in STEM areas, it is necessary to start encouraging them in 
a primary school. Opinions about STEM education and engagement are also influenced by transitions 
within education, from preschool to primary education and from primary to secondary education (Perry 
and MacDonald, 2015). Archer et al. (2012) in their research showed that the aspirations and attitudes of 
students aged from 10 to 14 years do not differ significantly. Due to the differences in previous research 
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results, it is necessary to investigate whether differences in age or attendance affect students’ attitudes 
about STEM studies. Based on this, the following hypothesis is set in the research: 

H2 — Age of the students influences the attitudes regarding the teaching within the framework 
ENABLE-BiH project 


Results 


Before testing the hypotheses, it is necessary to group the claims via EFA. Within EFA analysis, 
Varimax factor rotation and Kaiser Normalization were applied (Zhou and Li, 2023). The application of 
Kaiser Normalization determines the number of factors in the conducted EFA analysis. In order to group 
a statement into one factor, its value according to Kaiser Normalization must be greater than 1. When 
performing EFA analysis, it is necessary to meet certain conditions, namely: the correlation matrix is 
suitable for EFA analysis, which is proved by the Kaiser-Meyer-Olkin (KMO) indicator of sample adequacy 
and the sphericity of data by Bartlett's test is satisfied. The values of the KMO indicator should be greater 
than 0.6, while in Bartlett's test the significance level should be less than 0.05 (p <0.05) (PuSka, Maksimovi¢é 
and Stojanovié, 2018). The results obtained by EFA analysis showed that the sample is adequate (KMO 
= 0.894) and the data are spherical (p <0.000), which confirms this analysis. 

The obtained results of EFA analysis showed that the claims were grouped into five factors in which 
72.49% of the variance was explained. The obtained factors were used to examine the reliability of the 
measurement scale of the questionnaire via CA indicators. The value of this indicator should be greater 
than 0.70 (Al-Ansi, et al., 2023). 

The first factor covered seven claims and explained 22.781% of the variance (table 1). This factor 
seeks to explain the experience of students in STEM teaching and it provides answers to whether STEM 
teaching is better than traditional and whether students would still have STEM teaching. The highest 
grade for this factor was given to the statement “STEM teaching is useful for students” (average = 4.55), 
while the lowest grades were given to the statements: “I am proud to be part of STEM teaching” and “I 
would like to continue to have STEM teaching” (average = 4.14). The value of standard deviations (SD) 
showed that the highest variance in grades for the statement “I wish | still had STEM teaching” (SD = 
1,183), while the lowest variance in grades for the statement “STEM teaching is useful for students” (SD 
= 0.844). The obtained value of the CA indicator (CA = 0.897) showed that the measurement scales of the 
collected data are reliable for this factor. 

The second factor included six claims. This factor seeks to explain whether STEM teaching is 
interesting and fun for students and whether it allows them to better understand teaching material. The 
statement “I like examples in STEM teaching” received the highest grade for this factor (Average = 4.50), 
while the statement “STEM teaching interests me” received the lowest grade (Average = 4.31). The 
highest variance of grades is found in the statement “STEM teaching is more interesting than traditional” 
(SD = 1,083), while the lowest variance is in the statement “I like examples in STEM teaching” (SD = 
0,783). Measuring scales (CA = 0.835) were also found to be reliable for this factor. 

The third factor included three statements where the average values are between 4.33 and 4.42. 
This factor seeks to explain the usefulness of STEM teaching for students. Of these three statements, the 
greatest dispersion was in the grades of the statement “STEM teaching helps me master the material” 
(SD = 1,028), while the smallest dispersion in grades for the statement “STEM teaching makes learning 
easier” (SD = 0,898). The value of the CA indicator (CA = 0.82) confirmed the measurement scales. 

The fourth factor included the fewest claims of all factors, namely two claims in which the average 
score is uniform. This factor explains the experience of students with STEM teaching. The statement 
“My experience with STEM teaching is very good” (SD = 1,285) had a greater dispersion of grades. The 
performed CA analysis (CA = 0.838) confirmed the measurement scale for this factor. The fifth factor 
included three claims. This factor explains whether STEM teaching aids learning by better understanding 
the material. The statement “STEM teaching improves my understanding of the material” received the 
highest average grade (Average = 4.41), with the lowest variance in grades (SD = 0,920). The lowest 
average grade was given to the statement “STEM teaching helps me learn” (Average = 4.24), which had 
the highest dispersion (SD = 0,920). For this factor, the measurement scale for the questionnaire was 
confirmed (CA = 0.745). 
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Table 1. 
Factor and descriptive analysis of claims 

Gisins Factor hisan SD Cronbach's 
correlation Alpha 

Factor 1 - Explained variance = 22.781 

| am proud to have been a part of STEM classes 0.832 4.14 1.119 0.897 

| would recommend STEM classes to everyone 0.830 4.30 0.975 

STEM teaching is better than traditional teaching 0.748 427 ~~ (0.971 

| wish | still had STEM classes 0.725 4.14 1.183 

| wish there were more STEM classes 0.646 4.34 1.088 

STEM teaching is useful for students 0.603 4.55 0.844 

| enjoy STEM classes 0.597 4.20 1.052 

Factor 2 - Explained variance = 15.670 

STEM teaching allows connecting teaching material 0.702 4.46 0.822 0.835 

STEM teaching interests me 0.688 4.31 1.033 

The quality of STEM teaching is excellent 0.663 4.32 0.926 

STEM classes are more interesting than traditionalones §—(0..605 4,35 1.083 

| like examples in STEM classes 0.590 4.50 0.783 

STEM classes are fun 0.496 4.42 0.917 

Factor 3 - Explained variance = 12.547 

STEM classes help me master the teaching material 0.738 4.33 1.028 0.828 

In STEM teaching | understand the material better 0.691 4.34 0.906 

STEM teaching facilitates learning 0.658 4.42 0.898 

Factor 4 - Explained variance = 10.948 

| have a pleasant experience with STEM teaching 0.880 3.94 1.169 0.838 

My experience with STEM teaching is very good 0.791 3.95 1.285 

Factor 5 - Explained variance = 10.544 

STEM teaching helps me to learn 0.839 4.24 1.133 0.745 

| look forward to STEM continuing in the future 0.619 4.33 1.030 

STEM teaching improves my understanding of the material 0.607 441 0.920 


KMO = 0.894: Bartlett test = 1818.709; p < 0.000; Total explained variance = 72.490 


The study of the relationship between the factors was performed using the Pearson correlation 
coefficient. The results obtained by this analysis showed that for all factors there is a significant positive 
correlation at a significance level of less than 0.01 (Table 3). Factors 1 and 2 were the most closely related 
(r = 0.784; p <0.01), while factors 3 and 4 were the least related (r = 0.495; p <0.01) 


Table 2. 
Correlation between research factors 
Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 
Factor 1 1.000° 
Factor 2 0.784" 1.000° 
Factor 3 0.588" 0.749" 1.000° 
Factor 4 0.590" 0.594" 0.495" 1.000° 
Factor 5 0.697" 0.659° 0.650" 0.576" 1.000° 


Note: *. Correlation at a significance level of 0.01 


The research hypotheses were tested using the VRA (Oliinyk, 2023). The task of the VRA is to 
include several factors in the analysis to assess the impact of independent variables on the dependent 
variable (Famo and Machate, 2023; Stili¢, et al., 2023). The examination of the hypotheses was carried 
out by applying indifferent statistics at the significance level of 0.05 (Abdeldayem, Aldulaimi and Baqi, 
2022). Since two hypotheses have been set, two VRAs were applied. 

During the examination of the first research hypothesis, we started from the assumption that there 
is a significant difference in students’ attitudes in relation to the gender difference, and for that purpose, 
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a VRA model was created. The results of this analysis showed that there is no significant statistical 
difference in these attitudes (F-test = 1,779; p = 0.122), which provided a basis for rejection of the first 
hypothesis of the research. This VRA model explained 7% of the baseline (R2 = 0.070). Observing the 
individual influence of the factors, it was noticed that only with factor 5 there is a significant statistical 
difference (t-test = 2.128; p = 0.035), and this factor had the greatest influence on the direction of the 
regression function (Beta = 0.295). With two factors there is a negative influence on the direction of the 
regression function, namely with factor 1 (Beta = -0.192) and factor 3 (Beta = - 0.112). 

While we examined the second hypothesis, the research started from the assumption that the age 
of students influences attitudes about STEM teaching. For this purpose, two groups of students were 
formed. One group of students attends general classes and the other specialized classes. The results of 
the conducted VRA showed that there is a significant statistical difference between the observed groups of 
students (F-test = 11,088; p = 0,000), and thus the second research hypothesis was accepted. This VRA 
model explained 31.8% of the baseline (R2 = 0.318). Observing the partial influence of individual factors 
on the regression function, there is a significant statistical difference between three factors: factor 1 (t-test 
= 2,869; p = 0.005), factor 3 (t-test = -5,204; p = 0.000) and factor 4. (t-test = 2,780; p = 0.006). It should 
be mentioned that with factor 2 (Beta = -0.161) and factor 3 (Beta = -0.633) there is a negative influence 
on the direction of the regression function. The same results would be obtained by applying ANOVA 
analysis because hypotheses are tested in MRA in SPSS using ANOVA analysis (PuSka, Maksimomvi¢ 
and Stojanovic, 2018). 


Table 3. 
Hypothesis testing using regression analysis 
Independent variable: Gender difference Beta t-value p 
Factor 1 -0.192 -1.217 0.226 
Factor 2 0.071 0.404 0.687 
Factor 3 -0.112 -0.789 0.431 
Factor 4 0.136 1.162 0.248 
Factor 5 0.295 2.128 0.035 
Model summary: R = 0.264; R2 = 0.070; F-test = 1.779; significance = 0.122 
Independent variable: Age Beta t-value p 
Factor? (0,387 © 2.869 — 0.005 
Factor 2 -0,161 -1.068 0.288 
Factor 3 ; -0,633 © “5.204 0.000 
Factor 4 0,278 2.780 0.006 
Factor 5 0,012 0.102 0.919 


Model summary: R = 0.564; R2 = 0.318; F-test = 11.088; significance = 0.000 


Discussions 


In order to determine the existence of differences between students’ attitudes regarding STEM 
teaching which was implemented within the ENABLE-BiH project, which was assessed in the Public 
Institution “Ninth Elementary School’ Brcko District, data were first collected by students through a 
questionnaire. The questionnaire was distributed in two ways, physically and electronically, with paper 
and electronic questionnaires. It was observed during the data collection that a smaller number of blank 
questionnaires were for questionnaires distributed in paper form compared to questionnaires distributed 
in electronic form. The reason should be sought in the fact that students who received an electronic 
version of the questionnaire could open it on various media (Petrovic, 2014). Some students opened the 
survey, Started filling it out, and dropped out. Another disadvantage of the electronic survey is that one 
student can fill in the survey several times. However, in this research, access to the surveys was limited 
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with the IP address disabling that one student completes the survey more than once. In this way, this 
shortcoming was solved. 

Factor analysis grouped the data collected from students and the claims related to STEM teaching 
from the ENABLE-BiH project were grouped into five factors. These factors cover 72.49% of the variance 
that is common for social research (Puska, et al., 2021). Looking at the claims placed in the corresponding 
factors one can see there is a connection between these claims. In factor 1, all the claims are related to 
the intention to further use STEM teaching, that STEM teaching is useful and that students enjoy it. Factor 
2 grouped the claims concerning the interestingness, quality, and fun of STEM teaching. Factor 3 grouped 
the claims concerning understanding, mastery, and learning assistance by STEM teaching. Factor 4 
grouped the claims related to STEM teaching experience. Factor 5 included the claims regarding learning 
assistance, rejoicing in the continued use of STEM teaching, and the claim related to understanding the 
material. 

The obtained values showed that there is a positive significant correlation between all factors. The 
correlation for all claims ranges between 0.495 and 0.784, which is a very good correlation, especially 
when it comes to social research (PuSka, Maksimovi¢ and Stojanovic, 2018). However, this association 
does not mean that the gender difference and the age of students influence the observed factors. The 
results of the VRA showed that the age of students affects the observed factors while the gender difference 
does not. These results indicate that boys and girls equally accept STEM teaching and their attitudes are 
similar, thus confirming the results provided by Rosenzweig and Wigfield (2016). The attitudes they used 
are most similar in factor 2 because the significant value in VRA is the highest which shows that there is 
the smallest difference in attitudes in this particular case. With factors 1 and 3, there is a negative influence 
on the direction of regression analysis, which indicates that girls have better attitudes than boys regarding 
these two factors. This indicates that interest by girls has not decreased in STEM teaching, which does not 
confirm the results provided by Huppatz and Goodwin (2013). This finding is very positive, as it reduces 
gender discrimination in STEM teaching. If girls maintain their interest in STEM teaching, it is expected 
that there will be a larger number of women who will work in STEM areas in Bosnia and Herzegovina 
in the future. In this way, the justification of the ENABLE-BIH project in Bosnia and Herzegovina was 
confirmed, because this project delivered a positive effect on students in their early years and reduces 
gender discrimination in STEM areas. 

Observing the difference between the ages of students, the results of the VRA showed that there 
is a significant difference between the attitudes. However, by analyzing individual factors, one can see 
that in factors 2 and 3 there is a negative impact on the direction of regression analysis, which shows that 
students from specialized classes have better attitudes towards these factors than students from general 
class. Furthermore, the results for factor 5 indicate that there is no difference between the attitudes of 
these students. In this way, it was proven that students’ attitudes do not decrease with the age of students, 
and the observed students accept STEM classes from the ENABLE-BiH project equally well, thus denying 
the results from Bryan, Glynn and Kittleson, (2011). 

The findings from this study show that there is no gender discrimination in terms of acceptance 
of STEM classes’ from the ENABLE-BiH project. In addition, the results show that the interest is higher 
in certain segments of STEM teaching among students in specialized classes in relation to students 
in general classes. All mentioned point to the fact that the ENABLE-BIH project provided good results 
and it is necessary to implement more similar projects in order to improve the situation in Bosnia and 
Herzegovina in the STEM area. With the improvement of the STEM area, it is possible to achieve the 
growth and development of Bosnia and Herzegovina. 


Conclusions 


Education in STEM areas is key for the development of any country. Influencing the acceptance 
of STEM studies by students provides a good basis for continuing studies in these areas, which creates 
a favorable climate for development. This research was conducted in the Brcko District of Bosnia and 
Herzegovina and included students of the Public Institution “Ninth Elementary School”. The aim of this 
study was to examine the existence of differences in the gender and age structure of students. The results 
show that there is no gender difference in students’ attitudes towards STEM teaching implemented within 
the ENABLE-BiH project, thus rejecting the first hypothesis of this research. Furthermore, the results 
show that there is a difference in terms of student age, thus accepting the second hypothesis of this study. 
Further insight into the results of the VRA indicates that there is a difference between the attitudes of 
students, but there is no uniform difference. Some claims are better assessed by students from specialized 


www.ijcrsee.com 
482 


Puska, E. et al. (2023). Students’ attitudes about stem teaching case study from bréko district of Bosnia and Herzegovina, 
International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 11(3), 475-485. 


classes in relation to students from general classes and vice versa. In this way, it has been proven that 
students are equally interested in STEM teaching. 

The study findings show that the ENABLE-BIH project implemented in Bosnia and Herzegovina 
delivered good results, which was fully confirmed by this research. Therefore, in the future, more similar 
projects should be implemented in Bosnia and Herzegovina in order to influence students to continue 
their studies in these areas. Furthermore, the implementation of this project has shown that students 
need to get closer to the STEM area, by making a connection between these areas in teaching that 
is also enriched with practical exercises. In this way, students can understand these areas and have 
better attitudes regarding these STEM. This should affect the strengthening of STEM areas in Bosnia and 
Herzegovina and contribute to the development of the country. In the future, it is necessary to include 
more schools and students with similar projects in order to have a greater impact on the acceptance of 
STEM areas by students. 

This study has shown that innovations in education, in this case, the convergence of STEM areas 
through the connection of different areas, have a significant impact on the students’ acceptance of STEM 
studies. In future projects and research, it is necessary to use innovations in teaching in a similar way to 
improve students’ curiosity for STEM studies, which creates a precondition for strengthening these areas 
in certain countries. The scientific validity of the conducted research was to understand how elementary 
school students accept STEM subjects because it is very important to prepare students for these subjects. 
The social justification of this research is based on the preparation of students in elementary school for 
these subjects, so that they can continue their careers in them and so that there will be no shortage of 
occupations in the future. 
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Abstract: In this study, the influence of the application of an online teaching model on the development of digital 
cometences of future teachers in preschool institutions was examined. The research that this study deals with was carried out 
on a sample consisting of N=125 students of the 3rd year of basic academic and professional studies in the study program 
of teacher in preschool institutions. As a research instrument, a five-point Likert-type scale of attitudes was used. The results 
show that there is a statistically significant difference in the attitudes of future teachers in preschool institutions when it comes 
to self-assessment of digital competencies before and after the implementation of online classes during the state of emergency 
in Serbia during the Sars-Cov-2 virus pandemic. The conclusion of the study is that the implementation of online teaching in 
processing the content of the Methodology of getting to know the environment affects the improvement of the digital competences 
of future educators when it comes to: 1) creating digital resources; 2) application of digital technologies in immediate educational 
work; 3) the role of digital technologies in professional development. The results of this research can be used to encourage 
teachers to step out of the usual pre-pandemic practice and enrich their pedagogical work with combined teaching (hybrid 
model) where part of the material would be analyzed in person, and part would be analyzed online using web tools, websites 
and videoconferencing applications. 

Keywords: digital competences, online teaching, Methodology of teaching science. 


Introduction 


“Whether or not computers will enrich the learning environment of preschool children depends on 
the knowledge, skills and attitudes of preschool teachers. Colleges and universities for the education 
of future educators play an important role in providing opportunities for students to acquire appropriate 
knowledge, skills and positive attitudes towards the use of computers in preschool education” (Kutluca 
and Gokalp, 2011). In addition to compulsory and elective courses in the field of digital competence 
development attended at college / university, the indirect development of digital competence through 
various courses in which digital technologies are used to achieve the outcomes of these courses is of 
growing importance. 

Kalogiannakis (2010) indicates that there is a gap between the ICT subjects attended by future 
preschool teachers at the university and the expected level of ICT use in the environment of preschool 
education, and points out that adequate education of future preschool teachers during their studies is 
of crucial importance for the application of digital technology in preschool. Shortcomings in the use of 
digital technologies found in preschool are determined by the following factors: study plans and training of 
trainers (Romero-Tena et al., 2020). This is in line with the opinion of preschool teachers who recognize the 
educational level and training in the field of digital technology as the most important factors influencing the 
application of digital technology in practice (Liu and Pange, 2015), and believe that they need additional 
training in this field during their professional development (Mikeli¢ Preradovic, LeSin and Boras, 2017). 
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Training and professional development of educators, preparing them for the process of lifelong 
learning (Life Long Learning), monitoring trends and frequent changes in information and communication 
technologies is an unavoidable process that cannot be bypassed. The facts and results of research 
(published in the world and in our country) that indicate that the introduction of information technology 
into the preschool educational system is a response to the needs of children and in the interest of the 
child, represent one of the decisive driving factors for the active participation of educators in the process 
of integrating information and communication technologies in educational work. This determines the 
direction of the development of the program for the professional development of educators in practice, 
and certainly of students who are preparing for the profession of educators (Veli¢kovic, 2014). 

On the other hand, the authors point out the attitudes (and skills) of ICT use as important factors, 
which shape their use in the preschool context (McCarrick and Li, 2007). Attitude is considered one of the 
decisive factors in predicting an individual’s behavior (Tatalovi¢ Vorkapi¢ and Milovanovic, 2012). When 
it comes to the use of computers in preschool conditions, the attitude towards computers is defined as 
a “general assessment of a person or a feeling of acceptance or antipathy towards digital technology 
and computer-related activities” (Smith, Caputi and Rawstorne, 2000). Teachers who have more positive 
attitudes toward technology tend to integrate technology into their teaching (Marangunic and Granic, 
2015). Other authors emphasize the importance of the intention of teachers as a factor that directly affects 
the use of digital technology in practice (Gialamas and Nikolopoulou, 2010). The authors also indicate 
that, in addition to knowledge and skills, attitudes and intentions, the self-perception of the possibility of 
using digital technology in practice by teachers will directly affect their integration (Romero-Tena et al., 
2020). “Psychological context, especially of teachers’ perception, is necessary for understanding the use 
of digital technologies in their daily duties” (Wang et al., 2008) since the perceptions that teachers have 
about the use of digital technologies can significantly affect the practice in kindergarten. If teachers believe 
digital technologies to be useful, they are more likely to adopt and use them without any difficulty (Oldridge, 
2008). When a “new pedagogical approach or tool is presented, teachers make value judgments about 
whether that approach or tool is relevant to their goals. The more valuable the approach or tool is in their 
opinion, the more likely they are to use it” (Ottenbreit-Leftwich et al., 2010). Thus, although two teachers, 
after graduation, have the same level of knowledge and skills for the application of digital technologies 
in educational practice, positive / negative attitudes of educators and self-assessment will greatly affect 
the degree and quality of digital technology use in their future practice. However, if future teachers are 
provided with adequate training and practical application of digital technologies during their studies, it is 
likely that attitudes towards the use of computers in educational practice and their self-assessments will 
be more positive. 

Taking into account the above findings and the situation caused by the Covid-19 pandemic, which 
forced higher education institutions around the world to intensify the use of numerous digital tools for 
online learning in working with students, we conducted a study that examined and compared the attitudes 
of future teachers on self-assessment of digital competencies, before and after the implementation of 
online classes during the summer semester 2019/2020. 

Online teaching includes intentional and thoughtful support for student learning mediated by 
the Internet (Rapanta et al., 2020). Experiences of online learning through distance education can be 
asynchronous or synchronous. Asynchronous learning occurs when students can determine their own time 
to participate in learning through a variety of digital tools. Students can participate in communication and 
complete activities at a time of their choice and learn at their own pace. On the other hand, synchronous 
learning activities take place through live video and/or audio conferencing with instant feedback (Kim, 
2020). The effectiveness of online learning depends on the designed and prepared learning material, the 
engagement of the lecturer in the online environment and the interaction between the lecturer and the 
student (Bao, 2020). 

The transition from the usual way of working to online teaching, in response to the Covid-19 
pandemic, contributed to professors and students finding themselves in a new environment and seeing 
the differences between the previous way of working and teaching online, but also to the improvement 
of their digital competencies. Online teaching has set new requirements for professors and students: 
the use of various digital tools and resources and the application of new approaches in teaching and 
learning (KOnig, Jager-Biela and Glutsch, 2020). However, teaching and learning online is not entirely 
new for professors and students. Digitization in schools / universities began even before the pandemic, 
and it is reflected in, among other things, equipping “schools / universities with appropriate equipment, 
Striving to include ICT in curricula and giving students the opportunity to use advanced technological 
tools and digital resources for creative and innovative problem solving” (Kozma, 2011). The Covid-19 
pandemic has accelerated the digital transformation of higher education and introduced innovations in 
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higher education, which would normally take much longer, immediately after the closure of schools / 
universities. The digital transformation has brought several challenges, and one of the main issues is 
that it has caused mandatory modifications in the attitudes of online teaching administrators, teachers 
and students about the importance of the Internet for learning (Kozma, 2011). Research has shown that 
current students, as Gen Z representatives, want to learn new skills and gain more experience in the field 
of digital technologies (Kamarianos et al., 2020). Globally, student satisfaction with the organization of 
three segments of the teaching process was quite high and almost the same on all continents: for lectures 
3.30, tutorials and seminars 3.12, and mentoring 3.20 (Aristovnik et al., 2020). The authors believe that 
education, like all other segments of life and work, will never (and many should not) be the same, and 
that the crisis will have a lasting impact on learning, innovations and digitization (Aristovnik et al., 2020). 

The specificity of the situation in which we found ourselves due to the Covid-19 pandemic put the 
issue of digital competencies of students to the forefront, regarding the monitoring and implementation 
of online teaching, as well as the level and possibilities for the development of these competencies. Our 
interest in this paper is focused on examining the possibility of improving students’ digital competencies 
during the implementation of online teaching, indirectly, through the daily use of various digital tools for the 
purpose of learning, exchange and communication. Will students, future teachers, after three months of 
using digital tools for monitoring, implementation and evaluation of teaching have more positive attitudes 
about self-assessment of digital competencies and in which segments? 


Materials and Methods 


Research aim 

The aim of this paper is to recognize the change in the attitudes of future teachers on self- 
assessment of digital competencies under the influence of the implementation of online teaching, during 
the spring semester 2019/2020. Specifically, it was about reviewing the attitudes of future teachers on the 
self-assessment of digital competencies before the implementation of online classes and after the end of 
the semester. We wanted to find out whether there was a change in attitudes about self-assessment of 
digital competencies of future teachers under the influence of online teaching, which included the use of 
various web tools for communication and cooperation, creating digital content and its application during 
teaching. 

The research goal set in this way led to the following research task: Compare attitudes about 
digital competencies of future teachers in preschool institutions before and after the application of online 
teaching during the emergency situation in Serbia caused by the SARS-CoV-2 pandemic, through self- 
assessment of digital competencies. 


Sample 

The research sample consisted of third-year students of basic academic and vocational studies 
in the study program for the education of teachers in preschool institutions. The sample included N=125 
students (57 respondents were from vocational studies (colleges for education of preschool teachers) and 
68 from basic academic studies (study program Teacher in preschool institutions). 


Instruments 

The research presented in this study is a continuation of the research the results of which are 
presented in the paper published by Ceki¢-Jovanovic, Stepi¢, Mileti¢ (2020). Based on the results pre- 
sented in the mentioned published study, as well as on the basis of previously studied literature and 
research results of other researchers in the field of digital competencies and application of educational 
technology, for the purposes of this research, a research instrument that was used was an electronic 
questionnaire in the form of a five-point Likert-type scale that examined the attitudes and self-assessment 
of digital competence of future teachers in preschool institutions before and after the implementation 
of online teaching during the spring semester. This questionnaire was also used by Ceki¢-Jovanovi¢, 
Stepi¢, Mileti¢ (2020) in their research, and its reliability was assessed by calculating the Cronbach's a 
reliability coefficient which is a = 0,83, a>0,7. The first part included questions for collecting general data 
and information about the respondents (gender of respondents, their achievements in the Educational 
Methodology, the Environmental studies methodology, etc.). The second part was a five-point Likert- 
type scale, consisting of 26 statements (items). The statements in the questionnaire were formulated on 
the basis of the Rulebook on standards of competence for the vocation of preschool teachers and their 
professional development (Official Gazette of RS - Education Gazette, No. 16/2018), which is based on 
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DigComp 2.1. (Carretero, Vuorikari and Punie, 2017). The formulations of all statements are given in the 
study conducted by Ceki¢-Jovanovi¢, Stepic, Mileti¢ (2020), and they were used in this study as well, with 
minor changes. 


Procedure 

The beginning of the research presented in this paper deals with an initial examination of the 
attitudes of future teachers in preschool institutions (students of basic academic and vocational studies 
who are part of the study program Teacher in preschool institutions), through self-assessment of digital 
competencies before applying online teaching during an extraordinary situation. After the initial research, 
the students had online classes within the compulsory subject Environmental studies methodology 
for 3 months during the state of emergency in Serbia caused by the SARS-CoV-2 pandemic. Classes 
were mainly realized using the Google Classroom, video conferencing tool Zoom, Moodle platform and 
Edpuzzle application for editing interactive video materials. Students received teaching materials through 
these websites in the form of texts and video lessons, assignments were given via Google and Moodle 
platform, while they interacted with teachers and other students through discussions and analysis of 
teaching content via Zoom videoconferencing tool. 

The end of the semester and online teaching was followed by the final examination of the attitudes 
of students / future teachers in preschool institutions on the self-assessment of digital competencies after 
the application of online teaching during an extraordinary situation. The final examination of attitudes 
included the same research instrument described earlier. 


Analysis Procedures 

In order to compare the attitudes of future teachers in preschool institutions, it was first necessary 
to examine the normality of the distribution of data obtained by applying the research instrument (a 
questionnaire) within the initial and final survey. As the sample of respondents is greater than 50, the 
results of the Kolmogorov-Smirnov test given in Table 1 show that the significance is greater than 0.05 (p 
= 0.2), which means that there is a normal distribution of data. 


Table 1 
Test of normality of data distribution in the initial and final examination of the attitudes of future 
teachers in preschool institutions 


Respondent's code Kolmogorov-Smirnov2 Shapiro-Wilk 
Statistic df Sig. Statistic df Sig. 
The sum of the 1-initial examination 065 125 200° 981 125 074 
claims 2-final examination 085 125 026 922 125 ,000 


* This is a lower bound of the true significance. 


a. Lilliefors Significance Correction 


By further using the t-test, we examined whether there was a statistically significant difference 
between the results of the initial and final examination, and whether the attitudes of future teachers in 
preschool institutions differed significantly before and after the implementation of online teaching. 

If we summarize all the statements (Table 2), the t-test showed that the significance p=0.001 was 
less than 0.05, which means that there is a statistically significant difference in the results of the initial 
examination of attitudes compared to the final examination of attitudes, in favor of the final examination. 


Table 2 
T-test results 
t-test for Equality of Means 
t df Sig. (2- Mean Std. Error 95% Confidence 
tailed) Difference Difference Interval of the 
Difference 
Lower Upper 
The sum Equal variances -3,484 248 ,001 -4.656 1,336 -7,287 -2,024 
of the assumed 


claims 
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Results and Discussion 


In general, after the realization of online classes during the spring semester, students estimated 
that they had better digital competencies compared to the initial examination of attitudes. If we look at the 
results obtained for each individual statement (Table 3) within the assessment scale, the t-test shows that 
there is a statistically significant difference in favor of the results of the final survey. Significance is less 
than 0.05 when it comes to self-assessment of digital competences related to the following statements / 
items (IT7, IT12, 1113, 1715, IT17, IT20, IT22, 1725, IT26): assessment of the quality, relevance, accuracy 
and scope of digital information (IT7); use of appropriate digital technologies for designing and creating 
the necessary educational materials for working with children in the realization of the planned learning 
situation in the field of Environmental studies (IT12); application and integration of digital technology 
in the immediate educational work, in the realization of the planned learning situation in the field of 
Environmental studies (I1T13); multimedia presentation of content that is interactive and allows children 
to better understand and learn easier and faster in the field Environmental studies (IT15); application 
of digital technology that enables children to meaningfully use digital technologies for expression and 
learning through play within a topic or project in the field of Environmental studies (IT17); use of digital 
technologies that encourages children’s decision making and critical thinking within the educational work 
in the field of Environmental studies (IT20); adequate attitude and habit of using digital technologies (IT22): 
use of digital technologies for professional development of teachers (IT25): control of shortcomings and 
dangers of digital technology application (IT26). For all the mentioned items, after the realization of online 
classes, the students estimated that they had better digital competencies compared to the period before 
the realization of online classes. 


Table 3 
Differences in students’ attitudes 


t-test for Equality of Means 


95% Confidence 
Sig. | Std. Error | Interval of the Dif- 
df (2-tailed) ° Difference | ference 
I use e-mail in my everyday life and dur- | -1,532 | 219,57 094 
ing my studies. 
In my daily life and during my studies, I ,102 
3 


0 
I use social networks in my everyday life 2 0 
30 


and during my studies. 


I use digital technologies to exchange -1,243 
information 


I use digital technologies to work in dif- 


ferent bases oe 


use WWW - Internet databases. oe 


t 
783 
537 
682 

55 


= 
> 56 


I use digital technologies to summarize, 
compare and consolidate information 


; 082 
I use digital technologies to judge the -4,030 ; 
quality, relevance, accuracy and coverage 


4131 -,786 | -,269 


7 
5 
5 
6 
3 
of digital information 
1 


I successfully use digital technologies in | -,411 
finding and collecting 

I successfully use digital technologies in | ,2 2 5799 ,024 
planning 

I can effectively adapt, present and me- | -1,195 | 240,940 | ,233 

thodically transform 


sl2 
248 43 
248 99 
248 21 
248 ,49 
248 92 
248 
248 ,68 
48 79 


iS) 
ro) 
No) 


-,462 
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Iam competent to apply and integrate -2,599 | 248 ,010 -,264 ,101 -,464 | -,063 
digital technologies 

Iam competent to evaluate and docu- -,942 248 347 -,072 076 -,222 |,078 
ment in numerous ways 

Iam competent to present multimedia -4,322 | 193,14 -,568 -827 | -,308 
content that is interactive 


Digital tools enable children to access - eal 196 | 
information 


Iam competent in enabling children to _ 453 - aa - ea eal 55 
meaningfully use digital technologies for 

expression and learning through play 

Iam competent to enable children to use | -1,270 -,136 -346 | ,074 
digital technologies themselves to moni- 

tor their own learning 

I am competent to enable children to -,160 | 208 
progress at their own pace 

Modern technologies enable children to | -2,803 ce ,0 -,312 sl. -,531 | -,092 
make decisions and think critically 

I adequately and safely use the advan- - fea -,072 - (ieee 101 | 
tages of digital technology 

An adequate relationship and culture of cae 818 alt 10 -,328 - ea oo 
using digital technologies is necessary 

for every educator 

Iam competent to develop awareness -1,162 52 -,152 mil --409 |,105 
and habits for adequate use of digital 

technologies in children and parents 

Iam not competent enough to develop | -1,909 | 248 ,057 -,288 ,150 -,585 
awareness and habits for adequate use 

of digital technologies in children and 

parents. 

I believe that it is important that digital | -3,028 | 218,37 | ,003 -,320 ,105 -,928 |-,111 
technology is used for the professional 

development of educators. 


I believe that, as a future educator,Iam | -2,565 | 240,75 | ,011 -,360 ,140 -,636 | -,083 
not always able to control the disad- 

vantages and dangers of using digital 

technology. 


Further analysis of the research results shows that there is no statistically significant difference 
between the initial and final examination of the attitudes of future teachers in preschool institutions when it 
comes to self-assessment of digital competencies related to: the use of e-mail, WWW - Internet databases 
and social networks; the application of digital technology for information exchange; work in different 
databases for recording information about children, their parents, evaluation of educational work, etc.: 
summarizing, comparing and combining information from different digital sources; finding and collecting 
relevant information and educational materials for educational work in the field of Environmental studies; 
the use of digital technology in the planning of educational work in the field of learning Environmental 
studies; adaptation, presentation and methodological transformation in accordance with children’s age 
and the topic of educational work in the field of Environmental studies; application of digital tools for 
Organization and classification of information (charts, schemes, planners, schedules, mind maps, 
animations, video tutorials, etc.) in the field of Environmental studies; evaluation and documentation of 
data, photographs, digital texts, videos, etc.; the use of digital technologies as tools that enable children 
to access information during the implementation of educational work in the field of Environmental studies; 
enabling children to use digital technologies in monitoring their own learning; enabling children to make 
progress in educational work in the field of Environmental studies at their own pace, in a way that suits 
them best; adequate and safe use of the benefits of digital technology in educational work in the field of 
Environmental studies; developing awareness and habits in children and parents for the adequate use of 
digital technologies; 

The results indicate that the students similarly assessed their digital competencies before and after 
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the realization of online classes during the pandemic. And that online classes did not significantly affect 
their self-assessment of digital competence in the mentioned areas. 

The findings of one study indicate that self-assessments of digital competencies of future teachers 
in the areas of information and data, communication and collaboration and security are more than self- 
assessments in the areas of digital content creation and problem solving (Cebi and Reisoglu, 2020). 
The authors explain this by having the focus on theoretical knowledge in teacher training programs, 
lack of practice for digital content development and technical problem solving, and the fact that future 
teachers feel more advanced in these areas, probably due to the use of digital technology in everyday 
life in terms of these areas (Cebi and Reisoglu, 2020). We could say that the finding of our research is 
the opposite. During online classes, students had the opportunity to practically apply certain tools, to a 
greater extent than usual, and this resulted in a better self-assessment of digital competencies in the field 
of creating digital resources, direct work in educational practice and professional development. Analogous 
to the interpretation of the above research, we can assume that pedagogically designed use of digital 
technology during studies can lead to the improvement of digital competencies that are practiced during 
online classes. 

Adequate initial training in the field of digital competence development “makes students better 
perceive themselves in their development of critical thinking, problem solving and decision making, and 
points to the fact that this will increase their creative capacity and influence them to be more innovative, 
as shown by findings obtained in various studies” (Romero-Tena, 2019; according to: Romero-Tena et al., 
2020). 

Expressing positive attitudes / intentions is just one of the factors for successful integration of ICT, 
which has an effect on professional development and education of teachers (Gialamas and Nikolopoulou, 
2010), and, according to our results, vice versa. Shaping the pedagogical use of digital technology in 
working with students during online classes has proved important for improving attitudes about self- 
assessment of digital competencies. 

It should also be borne in mind that teachers’ pedagogical practices and behavior in practice are 
not static and are influenced by beliefs about the potential of ICT to improve learning as well as about the 
competence of teachers to integrate ICT into the curriculum (Webb and Cox, 2004). This means that even 
those teachers who have expressed positive attitudes or intentions may be disappointed and limit the use 
of digital technologies if they do not have continuity in professional development in this field (Gialamas 
and Nikolopoulou, 2010). This tells us that when it comes to application of digital technology, we need 
adequate and continuous support for students, future teachers, but also for practice teachers. 


Conclusions 


Based on the results presented in this study, it can be concluded that the implementation of online 
classes with students of basic academic and vocational studies who are part of the study program 
Teacher in preschool institutions has a positive effect on their attitudes and self-assessment of digital 
competencies. 

Comparing the results obtained during the initial and final examination of students’ attitudes, it can 
be seen that students believe that after the implementation of online classes they have more developed 
digital competencies related to: 1) creation of digital resources; 2) application of digital technologies in 
the immediate educational work and 3) role of digital technologies in professional development. Overall, 
differences between the initial and final measurements were not observed in the following fields: 1) use of 
digital technologies in everyday life; 2) work with information; 3) planning, monitoring and documenting; 
4) respect for children’s characteristics; 5) safe use of digital technologies. 

These results tell us that practicing the use of digital tools during online classes has helped future 
teachers feel more confident in creating digital resources and applying digital technologies for chidren’s 
learning and development, but, at the same time, they have become aware that they are not competent 
enough to assess the quality, relevance, accuracy and coverage of digital information and that they cannot 
always control the shortcomings and dangers of digital technologies. 

When implementing online teaching or hybrid models in post-pandemic conditions, it is necessary to 
keep in mind the potential effects of certain objective and subjective factors that discourage teachers from 
opting for this way of working. The absence of adequate technical conditions for the implementation of 
teaching activities can be a significant objective factor. When it comes to subjective factors are concerned, 
they are mostly reflected in the approach of teachers and students. Hybrid and online teaching require 
highly motivated and creative teachers who will have to invest more time and effort to create different 
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learning activities and materials than those to which they and students were used to before the pandemic. 
Although web applications, such as Edpuzzle, Google Classroom, Zoom, etc., are quite easy to use, 
teachers, especially those with insufficient experience in the application of modern technology, may have 
difficulty with their implementation. Additional training, professional development and systematic support 
for the development of teachers’ digital competencies would certainly contribute to a more successful 
implementation of hybrid teaching in post-pandemic conditions. 

The results of this research can be used to encourage teachers to step out of the usual pre- 
pandemic practice and enrich their pedagogical work with combined teaching (hybrid model) where part 
of the material would be analyzed in person, and part would be analyzed online using web tools, websites 
and videoconferencing applications. This would certainly affect the improvement of digital competencies 
of students - future teachers in preschool institutions. The results of this study can also serve as a starting 
point for other researchers who want to examine this issue in more detail and more comprehensively, 
taking into account the possibility and need for online learning outcomes to be tested in different working 
conditions, learning environment, within other subjects, at other levels of education, on a different and 
larger sample of respondents, and so on. 
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Abstract: This research analyzed the opinions of primary school teachers and students regarding the benefits of outdoor 
education, using the example of School in Nature, for the organization of educational activities within the instruction process. 
It explored educational, social, recreational, and ecological aspects, as well as the existing differences in their opinions. The 
research was conducted on a sample of elementary school teachers and fourth-grade students in the Bor District in the Republic 
of Serbia. The research involved 223 participants (elementary school teachers N=73, and fourth-grade students N=150). A 
questionnaire was designed for the purposes of this research to assess the opinions of teachers and students towards the 
learning environment and the effectiveness of implementing outdoor education in the context of School in Nature. The research 
results indicate a compliance between teachers’ opinions about the significance of outdoor education, especially through School 
in Nature, and its benefits to the educational aspect of the teaching process. Students displayed positive attitudes related to 
social interaction in learning, emphasizing the development of positive social relationships and skills. Differences in opinions 
between teachers and students regarding the learning environment and the effectiveness of outdoor education in meeting 
learning objectives represent a significant starting point for better understanding the quality of the teaching process. 

Keywords: outdoor education, School in Nature, learning environment, student, teacher. 


Introduction 


Outdoor education, having the forms of field trips or experiential learning, is becoming an 
increasingly significant component of education systems worldwide. It provides students with a unique 
opportunity to expand their knowledge and levels of understanding through practical experiences outside 
the classroom. Outdoor education represents an innovative approach to education that goes beyond the 
traditional inclass learning. This learning method involves taking students outside the school premises to 
gain practical knowledge, explore nature, and apply acquired knowledge in real-life situations. Outdoor 
education utilizes interactive and experiential learning methods to foster engagement, motivation, and a 
deeper understanding of learning topics. Conducting lessons outside the classroom, as a specific aspect of 
the teaching process, allows for a direct connection between the school and the real world, simultaneously 
stimulating motivation and curiosity among students, while providing teachers with the opportunity to 
change their practices (Rickinson et al., 2004; Andelkovi¢ and Mihajlovic, 2012; Radivojevi¢ and Jeli¢, 
2016; Ristié and Blagdani¢, 2017; Enachescu, 2019; Milosavljevic Dukic, Bogavac and Stojadinovic, 
2022; Mitic, Dejkovic and Zlatanovi¢ Markovic, 2022; Begic, 2022; Borsos, Fekete and Boric, 2023). 

The primary goal of outdoor education is to provide students with optimum learning conditions 
directly derived from experience, which can be a powerful tool for developing critical thinking, teamwork, 
problem-solving skills, and emotional intelligence (Auer, 2008). Outdoor education also provides an 
opportunity for the integration of different subjects and disciplines, allowing students to connect various 
aspects of knowledge and understand their application in the real world. Experiences gained through 
outdoor activities compel teachers to integrate activities inside and outside the classroom to create a 
favorable environment for the education process. These activities that students experience, both inside 
and outside the classroom, contribute to a sustainable process of learning and development (Kocayigit 
and Ekinci, 2020). Authors Andié and Mazar(2023) conclude that the role of a teacher, including their 
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practices, emotion management, personality, and competencies, is of exceptional importance for 
achieving learning objectives in school subjects, as well as in interdisciplinary topics, such as sustainable 
development curriculum (Andi¢ and Mazar, 2023). 

An interdisciplinary approach to learning promotes the holistic development of students, fostering 
creativity, curiosity, and self-confidence, with the integration of different fields of knowledge being one of 
the key objectives of the holistic learning process. In achieving this goal, outdoor activities play a significant 
role, allowing students to connect theoretical knowledge with real-life experiences and application in the 
natural learning environment (ileritiirk, 2023). The planning of these activities should be an integral part 
of the teaching process. 

Findings from some research indicate a possible positive impact of spending time in nature on 
students’ cognitive functions and their interest in learning. Being in nature enhances students’ collaboration 
and problem-solving skills while simultaneously increasing their happiness, well-being, and motivation for 
learning (Manni et al., 2013; Capaldi, Dopko and Zelenski, 2014; Yli-Panula, Jeroneni and Lemmety, 2019). 
According to relevant studies, this effect can result in increased student engagement in the classroom, 
which has the potential to enhance their learning achievements. This leads to the conclusion about the 
significance of incorporating natural elements into the education process as a potentially beneficial factor 
for supporting cognitive functioning and student learning. 

The results of the research conducted by the group of authors (Kuo, Browning and Penner, 2018) 
describe the practice of supporting and encouraging students during the learning and development 
process, which they refer to as “Refueling Students in Flight.” In their findings, the authors emphasize 
the importance of providing continuous support through natural resources to enable students to progress 
and achieve their academic and personal goals. In this research, the hypothesis that outdoor lessons 
have positive, not negative effects on subsequent classroom engagement was tested. It was found 
that classroom engagement was significantly better after outdoor lessons than after regular lessons. 
Such results suggest that outdoor lessons can actually motivate students to engage more in regular 
lessons, even when they explore content that is not directly available to them. This practice, referred to as 
“Refueling Students in Flight,” advocates the inclusion of more outdoor lessons in formal education (Kuo, 
Browning and Penner, 2018). 

One of the most well-known forms of outdoor education is the “School in Nature,” where students 
spend time in nature and actively participate in various educational activities. The organization of this form 
of outdoor education aims to acquaint students with objects, phenomena, processes, and achievements 
in nature. “School in Nature,” as a form of outdoor education, carries significant educational and 
developmental importance because it establishes the most effective connection between the teaching 
process and the natural laws and phenomena. Through the implementation of the “School in Nature,” 
teaching is efficiently linked to current social and life contexts, contributing to comprehensive development 
and the improvement of students’ personalities, providing an opportunity for a deeper understanding and 
practical application of acquired knowledge. 

The perspectives of teachers and students play a crucial role in shaping and assessing the success 
of any educational approach, including outdoor education. The teacher’s perspective involves planning 
and organizing field activities, adapting teaching methods and materials, and managing the group of 
students. Teachers are essential for the successful implementation of outdoor education because they 
are responsible for ensuring the safety of students, motivating learning, and maximizing the benefits of 
the field experience. They must be experts in their subject areas but also need to be capable of adapting 
to changing circumstances and student needs during the field trip. On the other hand, the perspective of 
students is crucial for understanding the impact of outdoor education on their development, motivation, 
and engagement. Through the “School in Nature,” students gain practical experiences that stimulate 
their curiosity, creativity, and self-confidence. Through outdoor education, students have the opportunity 
to apply acquired knowledge and skills in the real world, develop interpersonal and teamwork skills 
through collaboration with peers and teachers, and form an awareness of nature and the environment, 
understanding the importance of preserving natural resources. 

The competencies required of teachers for outdoor activities are dynamic and complex due to 
the close connection to the pedagogical, psychological, methodological, didactic, and subject-specific 
competencies of teachers. As such, these activities encompass the following: fostering students’ 
creativity, critical thinking, good communication, innovation, research, problem-solving, and environmental 
awareness. In the outdoor environment, the teacher should provide individual support to students, motivate 
them to take charge of their own learning, support different learning styles, provide appropriate feedback 
on their work, and ensure a safe learning environment. 

The dedication teachers invest in the planning and implementation of learning material often dictates 
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the impact outdoor education will have on students. Linking outdoor education with classroom concepts 
implies that teachers adapt experiential learning in informal settings to stimulate interest, motivation, anda 
desire for exploration and knowledge acquisition in students. This will enable the improvement of learning 
in a formal context. If classroom education is enhanced with one of the forms of outdoor education, such 
as teaching in the schoolyard, visits to natural, cultural, sports, and economic facilities, “School in Nature,” 
field trips and so on, and if it is done with the intention of integrating curriculum content, where multiple 
subjects are simultaneously organized, students will better understand the content, actively engage in the 
learning process, take responsibility for self-learning, etc. The essence of the mutual complementarity of 
classroom and outdoor education and their integration into a unified and high-quality education process is 
that teachers and students nurture and create pedagogical, educational, and learning strategies that are 
occasionally realized in nature (Jurcic, 2015). 

The research in this paper is focused on examining the opinions of teachers and students in 
elementary schools regarding the importance of implementing “School in Nature” as a form of outdoor 
education, concerning its educational effectiveness, the quality of organization, and the aspect of 
“innovation” in the teaching process. The quality of education is determined by its structure, learning 
materials, teaching methods, and the ability to direct learning towards concrete interaction with the 
environment. The aspect of “innovation” is reflected in the cognitive, psychological, and geographic 
spheres (Orion and Hofstein, 1994). The significance of the “School in Nature” is evident in the realm of 
practical and empirical application of acquired knowledge, the development of social relationships, as well 
as in the sports and recreational aspect. “School in Nature” represents an understanding of the educational 
and pedagogical significance and its contribution to increasing students’ academic achievements. As a 
form of outdoor education, it is pedagogically grounded and interesting to students in learning of subject 
matter content. What is most remembered aspect in adulthood, related to a school field trip, is the escape 
from the classroom routine (Pace and Tesi, 2004). 

The research conducted is significant in the field of learning environments because there are very 
few studies that pertain to the effectiveness of the outdoor learning, emphasizing the natural environment 
and practical education, which provides a positive learning environment and the study of the Nature and 
Society subject. Research shows that teachers often avoid outdoor learning because “School in Nature” 
is not a pleasant experience for them (Fido and Gayford, 1982). 

However, many researchers believe that the environment to which students are exposed and how 
students feel in the classroom also influence their interest in the learning process (Fraser, 2012). As a 
result of exposure to a favorable learning environment, students achieve higher academic performance 
(Fraser, Fisher and McRobbie, 1996). One of the main shortcomings mentioned is the factor of “novelty.” 
Students may become distracted when exposed to a new environment, spending too much time adapting 
to the new surroundings, which can deter them from actual learning. An analysis of studies has shown 
that organized education activities in nature are a powerful tool that helps improve learning performance, 
social, emotional, and personal development, contributing to the preservation and enhancement of 
students’ health. Additionally, outdoor learning activities can be attributed to active teaching methods, 
and at the same time, outdoor education provides a favorable psycho-emotional space for the mental, 
physical, social, and moral well-being of students. 


Materials and Methods 


The aim of the research is to examine the opinions of elementary school teachers and students 
regarding the importance of outdoor education, specifically through the example of the School in Nature, 
in organizing learning activities and meeting learning objectives. The research seeks to identify their 
opinions towards the benefits of outdoor education (School in Nature) and various aspects of the teaching 
process, such as the educational, social, recreational, and ecological aspects, and finally to identify any 
differences in the opinions of teachers and students. The operationalization of the objective was carried 
out through the following tasks: 

1. Examine teachers’ opinions on the importance of outdoor education, especially through the 
example of the School in Nature, in various aspects of the teaching process (educational, social, 
recreational, and ecological). 

2. Investigate students’ opinions regarding the benefits of outdoor education, especially through 
the example of the School in Nature, in various aspects of the teaching process (educational, social, 
recreational, and ecological). 

3. Identify the existing differences in the opinions of teachers and students regarding outdoor 
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education, especially through the example of the School in Nature, in organizing learning activities and 
meeting learning objectives. 

The initial research hypothesis is as follows: Teachers and students recognize the importance 
of outdoor education, especially through the application of the School in Nature, in organizing learning 
activities and meeting learning objectives. 

The following auxiliary hypotheses have been proposed: 

h1: There is compliance between teachers’ opinions regarding the significance of outdoor education, 
using the example of School in Nature, and its benefits for the educational aspect of the teaching process. 

h2: There is compliance between students’ opinions regarding the significance of outdoor education, 
using the example of School in Nature, and its contribution to various aspects of the teaching process, 
particularly in the social and recreational aspects. 

h3: There are statistically significant differences in the opinions of teachers and students regarding 
the contribution of outdoor education, using the example of School in Nature, to the realization of learning 
activities within the instruction process, with the expectation that teachers will express a higher degree of 
recognition of the importance of outdoor education compared to students. 

The research was conducted on a sample of primary school class teachers and fourth-grade 
students in the Bor district in the Republic of Serbia. A total of 223 participants took part in the study, with 
73 primary school class teachers (N=73) and 150 fourth-grade students (N=150). Out of the 73 surveyed 
primary school class teachers, 43 (58.9%) were from urban schools, and 30 (41.1%) were from rural 
schools. Among the 150 students, 88 (58.7%) were from urban schools, and 62 (41.3%) were from rural 
schools. 

In the study, a specially designed Likert-type questionnaire was used, where participants rated their 
agreement on a scale of 1 to 5, with 5 being “completely agree”, 4 being “agree”, 3 being “undecided”, 
2 being “disagree”, and 1 being “completely disagree”. This questionnaire was designed to assess the 
Opinions of teachers and students regarding their attitudes towards the learning environment and the 
effectiveness of implementing outdoor education, with a focus on the School in Nature program. Numeric 
data were presented using mean, standard deviation, as well as minimum and maximum values. Student's 
t-test was used to examine the differences between teachers and students. Statistical significance was 
defined at a probability level of p < 0.05. Statistical analysis was performed using the SPSS software 
version 25. 


Results 


The first research objective aimed to investigate teachers’ opinions regarding the importance 
of outdoor education, particularly through the School in Nature program, in various aspects of the 
educational process (educational, social, recreational, and environmental). Teachers expressed their 
level of agreement with statements about the educational, social, recreational, and environmental aspects 
of the education process in the context of the School in Nature on a five-point Likert scale. The results of 
the study are shown in Table 1. 

Teachers believe that outdoor education, such as the School in Nature program, has the most 
significant impact on the development of positive social relationships among students (M=4.6). It also 
contributes to increased student achievements (M=4.1) and encourages students to engage in recreational 
activities (M=3.8). However, ithas the least impact on the development of students’ skills (M=3.7). Research 
findings obtained during the survey of teachers’ opinions about the influence of outdoor education within 
the educational and pedagogical aspects indicate that teachers believe that School in Nature has the 
biggest effect on the more successful organization of the teaching process and education in general 
(M=4.4). These research findings are corroborated by teachers’ opinions that the learning environment 
provided by the School in Nature contributes to students’ more successful mastery of the curriculum 
(M=4.2). Teachers assess that the practical and empirical learning space within the School in Nature 
particularly contributes to the more successful implementation of the Natural Sciences and Society subject 
content (M=4.0). They also recognize the significant effectiveness of outdoor education in delivering the 
curriculum of all subjects (M=4.1). In connecting outdoor education with the classroom concept, teachers 
see a high contribution in acquiring knowledge and competencies related to environmental conservation, 
social community, culture, and the environment that the School in Nature provides (M=4.1). Teachers 
believe that students’ engagement in choosing the destination mostly affects the effectiveness of outdoor 
educational activities (M=4.5). Moreover, students’ desire for exploration and knowledge acquisition 
in natural settings is encouraged by the formal classroom learning enhanced with destination-related 
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content, providing students with essential knowledge of concepts related to interacting with the natural 
setting of the selected destination (M=4.2). Teachers emphasize the importance of planning experiential 
learning within outdoor education and the alignment of the School in Nature programs with subject 
content (M=4.0). The research findings indicate that teachers are aware of deficiencies in the preparation 
and implementation of outdoor education (M=1.6) and believe that teacher training, encompassing the 
planning and execution of the School in Nature, would be highly beneficial for future educational work in 
the classroom (M=4.0). 


Table 1 
Teachers’ opinions on the significance of outdoor education in terms of the educational, social, 
recreational, and ecological aspects 


N Min | Max M sD 
_ Contribution of outdoor education / School in nature to | 
_ the: 
_ more successful mastery of the curriculum by students 73 3.00 5.00 4.260 0.624 


_ more successful implementation of the Nature and Society 
; 73 3.00 — 5.00 4.041 0.676 
_ subject 

_ more successful implementation of all subjects’ curriculum 73 2.00 5.00 4.110 1.021 
_ more successful organization of the education process and 


a 73 300 5.00 4.479 0.669 
_ instruction in general 


_ improved student achievements 73 3.00 | 5.00 4.164 0.817 
_ development of positive social relationships 73 4.00 5.00 4.685 0.468 
_ development of students’ abilities 73 2.00 | 5.00 3.781 0.989 
"encouragement of recreational activities = 73 «200 «=S«5.00s«i384Dst«éBF 


- acquisition of knowledge and competencies about nature 
; ; : 73 3.00 5.00 4.150 0.739 

_ conservation, community, culture, and the environment 

Planning and preparation for outdoor instruction: 
_ Knowledge about the destination 73 3.00 | 5.00 4.233 0.773 
Previously acquired knowledge about the basic concepts re- | 
oe 73 3.00 | 5.00 4.247 0.703 

_ lated to the destination | 
_ Student involvement in choosing the destination 73 3.00 © 5.00 4521 0.669 


| Alignment of the School in Nature program with the subject 
73 3.00 | 5.00 4.041 0.676 


content 

Teacher education in the field of planning and implementation 65 3.00 | 5.00 4.061 0.845 
| Limitations: | 

Preparation 73 4.00s3.00—«*S2* 71 0.625 
_ Implementation 65 1.00 3.00 1.692 0.727 


N=number of respondents, Min=minimum, Max=maximum, M=Mean, SD=Standard deviation 


The second research objective aimed to examine the opinions of students regarding the significance 
of extracurricular activities through educational, social, recreational, and environmental aspects of the 
educational process. Students expressed their agreement or disagreement on a five-point Likert scale 
with statements related to the School in Nature. The results of the research are presented in Table 2. 

Students find classes conducted outside the classroom very interesting (M=4.6) because they 
believe they learn more than in regular classroom lessons (M=4.5). Research findings on students’ 
Opinions regarding the contribution of School in Nature show that students support activities conducted 
outside the classroom because they actively engage in the learning process when conducting research to 
solve problems (M=4.2) related to various educational content (M=4.2). From the students’ perspective, 
the contribution of outdoor education is primarily associated with the development of positive social 
relationships among students (M=4.2) and the encouragement of sports and recreational activities 
(M=3.3). Lower scores are recorded in terms of students’ assessments related to the contribution to 
increasing student achievements (M=2.1) and the development of students’ abilities (M=2.3). Students 
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see the importance of School in Nature in mastering subject matter through concrete activities (M=4.1). 
According to students, the effectiveness of the outdoor environment, which emphasizes the natural 
environment, is reflected in the fact that it allows them to practically form a relationship with nature and its 
values, the social community, culture, and the environment (M=3.8). Students believe that the practical 
and empirical application of acquired knowledge is reflected in the way they can more easily connect 
the phenomena and objects they encounter in the natural environment in an interesting and engaging 
manner (M=3.5) and relate them to multiple subjects (M=3.4). Students assess that outdoor education 
helps them in mastering the subject matter of Nature and Society (M=3.1) because concrete activities 
serve as inspiration (motivation) for further research in the subject and natural phenomena (M=3.0). 
Students have a limited perception of the complementarity between classroom and outdoor education, as 
evidenced by the lower scores related to their assessments of success in acquiring educational content 
in the classroom (M=2.8) and active participation in teamwork (M=2.8). Their interest in the learning 
process outside the classroom is minimal, as indicated by the research results of student perceptions of 
School in Nature regarding the clarity of educational content after its practical application (M=2.7) and 
the efficiency in solving tasks in a natural environment (M=2.2). Students believe that there is a need for 
alignment between the School in Nature program and the subject content (M=3.5) to improve awareness 
of the destination aimed at forming initial attitudes towards learning outside the classroom (M=2.4) and 
increasing their interest in acquiring basic knowledge related directly to educational tasks (M=2.0). 


Table 2 
Students’ opinions about the importance of outdoor education 
N Min Max M SD 
The benefits of outdoor education: | 
Mastering subject matter through concrete activities 150 2.00 5.00 4100 | 1.002 
Mastery of subject matter in Nature and Society through 450 1.00 5.00 3.187 4.083 
research 


Getting to know phenomena and objects in an interesting 450 2.00 5.00 3.507 0.995 
and engaging way 

Success in acquiring educational content in the classroom 150 1.00 5.00 2.867 | 0.748 
Clarity of educational content after its practical application 150 1.00 5.00 2.193 0.922 
Inspiration (motivation) for further exploration of subjects 450 2.00 5.00 3.060 0.869 
and natural phenomena 

The class is interesting when conducted outside the 
classroom 

| learn more outside the classroom than during regular 
school lessons 

Educational content encountered in the natural environment 
can be more easily connected across multiple subjects 


150 4.00 5.00 4.600 0.492 
150 4.00 5.00 4.547 | 0.499 


150 2.00 5.00 3.413 © 0.921 


Active participation in teamwork 150 1.00 5.00 2.833 1.089 
Outdoor education for various educational content 150 3.00 5.00 4.207 0.745 
Efficiency in solving tasks in a natural environment 150 1.00 5.00 2.260 | 1.045 


Attitude towards nature and its values, the social community, 


: 150 2.00 5.00 3.886 | 0.952 
culture, and the environment 


Conducting research to solve problems 150 3.00 5.00 4.253 | 0.697 
Increasing student achievements i i's <st i stti(‘<i‘i;‘;*é«*wSO;CtC“‘z|OO}| hUC8OOO CO 193—Sts«d*CO 
Developing students’ abilities 150 1.00 5.00 2.353 1.024 
Developing positive social relationships among students 150 3.00 5.00 4.293 0.719 


Encouragement of sports and recreational activities 150 1.00 5.00 3.307 1.158 
Organization of outdoor education 

Awareness of the destination for forming initial attitudes 

towards learning outside the classroom 

Alignment of the School in Nature program with subject 

content 

Basic knowledge of concepts directly related to educational 

tasks 


150 1.00 5.00 2413 1.205 
150 2.00 4.00 3553 = 00.596 
150 1.00 5.00 2.027 1.198 
N=number of respondents, Min=minimum, Max=maximum, M=Mean, SD=Standard deviation 
The third research task focused on examining the differences in the opinions of teachers and 


students regarding the significance of outdoor education, using the example of School in Nature. The 
attitudes of teachers and students were assessed concerning the organization, planning, alignment of 
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program content, as well as their perceptions of the educational and pedagogical effectiveness in terms 
of the contributions to the educational process and the quality of the educational process. A T-test for large 
independent samples indicates that there is a statistically significant difference in the perception of certain 
elements of the importance of outdoor education, as exemplified by School in Nature, between teachers 
and students. The research results obtained are presented in Table 3. 

Statistically significant differences were identified in the opinions of teachers and students 
regarding organization, planning, and alignment of curriculum. Teachers expressed their competence 
for conducting outdoor education by stating that geographical (4.49+0.50), intellectual (4.40+0.66), 
and psychological (5.00+0.00) preparation is necessary to provide a positive learning environment in 
School in Nature. In contrast, students gave lower scores, p=0.000. In the assessments by students, 
there is a prevailing attitude compared to teachers related to the evaluation process after the activities 
in School in Nature. According to students, this evaluation is necessary to understand the significance 
of outdoor education in achieving the educational and pedagogical goals in the area of pragmatic and 
empirical application of acquired knowledge and experiences (1.97+0.75), p=0.000. Within the sphere of 
educational and pedagogical effectiveness in the educational process, teachers emphasize the cognitive 
domain (knowledge acquisition) more (3.88+0.76) compared to students (3.19+1.08), p=0.000. However, 
for students, the affective domain (attitudes, values, interests) is of greater importance (4.25+0.70) 
compared to teachers (3.92+0.95), p=0.000. In the psychomotor domain (skills, abilities), both teachers 
and students have similar preferences (aspirations) to be achieved in the outdoor education environment, 
p=0.716. Teachers recognize the educational and pedagogical significance of School in Nature as a form of 
outdoor education and its connection to the classroom concept through experiential learning in formal and 
informal settings (4.01+0.77). In contrast, students’ interest in the learning process is significantly lower 
(2.05+1.02), p=0.000. The outdoor education environment, which involves organizing the educational 
process through participant interaction and implementing content, is equally acceptable to both students 
(4.60+0.49) and teachers (4.48+0.67), p=0.130. Regarding the perceived contribution of School in Nature 
and its educational and pedagogical significance, a statistically significant difference is observed, with 
teachers recognizing its importance more than students, especially concerning students’ educational 
achievements through content correlation between subjects (4.11+0.94) and the functionality of acquired 
knowledge (4.47+0.69), p=0.000. The research findings show that there are differences in the opinions of 
teachers and students when it comes to educational effectiveness and the quality of School in Nature’s 
organization. Teachers more strongly emphasize the aspect of “innovation” in knowledge acquisition 
through outdoor education compared to the traditional classroom approach (4.41+0.66), p=0.000. 
Regarding the essence of the complementarity between classroom and outdoor education, as exemplified 
by School in Nature, teachers recognize it more than students, especially in terms of enhancing the quality 
of the educational process through experimentation and research (3.73+1.19) and in the higher quality 
of pedagogical work in activities that provide enjoyment and interest (3.96+0.77), p=0.000. A statistically 
significant difference can be noticed in the opinions of teachers and students regarding the contribution 
of educational and pedagogical activities in nature, which provide a favorable time for relaxation from the 
classroom routine and the improvement of social skills. Teachers emphasize the social (4.42+0.69) and 
adventurous aspects (4.23+0.77) of outdoor education more than students, as well as the environmental 
aspect (4.25+0.70), p=0.000. Regarding the sports and recreational aspect, both teachers and students 
believe it is a powerful tool that contributes to the preservation and enhancement of students’ health, 
so there is no statistically significant difference in the attitudes of teachers and students on this aspect, 
p=0.701. The implementation of inclass and extracurricular activities outside the classroom, as exemplified 
by School in Nature, is a specific form of work where both teachers and students nurture and create 
situations and strategies for educational and pedagogical work in natural conditions. Students emphasize 
the attitude of the “use-fulness” of outdoor education (2.61+1.20) more than teachers do, p=0.000. 
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Table 3 
Differences in the opinions of teachers and students regarding the benefits of outdoor education 
The attitudes of teachers and students teachers students 
M sD M SD T df p 
Organization, planning, alignment of 
curriculum content 

Geographical preparation 449 0.50 235 066 24660 221 0,000 
lintellectual preparation 440 0.66 2.07 0.81 21.391 221 0,000 
Psychological preparation 5.00 0.00 455 0.50 7642 = 221 0,000 
Assessment 1.00 0.00 197 075 11.064 221 0,000 

Contribution in the spheres of the teaching 

process 

Cognitive domain — knowledge acquisition 3.88 0.76 3.19 1.08 4.884 221 0,000 
Affective domain — attitudes, values, interests 3.92 0.95 425 0.70 2.977 221 0,003 
Psychomotor domain - skills, abilities 3.37 1.33 3.31 1.16 0.364 221 0,716 


Quality of the teaching process 
Connecting outdoor education with the classroom 
concept — experiential learning in formal and 401 0.77 2.05 1.02 14.566 221 0,000 
informal settings 
Organization of the teaching process through =| 44g | oe7 =| 460 | 049 | 1520 | 21 | 0130 
participant interaction and content implementation 
"Content correlation between subjects 4.11 0.94 2.87 0.75 10.699 221 0,000 
Functionality of acquired knowledge 447 0.69 3.06 0.87 12.094 221 0,000 
“Innovation — acquiring knowledge through outdoor 
education compared to the traditional classroom 4.41 0.66 3.41 0.92 8.267 =. 221 0,000 
approach 
Quality of the educational process — 373-149 196 0.74 13552 291 0,000 
(experimentation and research) 


Quality of pedagogical work (enjoyment and 396 077 169 063 93.281 994 0,000 


interest) —_ . — 

Social aspect (improvement of social skills) 442 0.69 2.25 0.70 21.948 221 0,000 
seh aspect (a break from the classroom 423 OTT 2031.20 14338 291 0,000 
Environmental aspect 425 0.70 241 = 1.21 12.032 221 0,000 
Sports and recreational aspect 4.15 0.74 410 1.00 0.384 221 0,701 
Domain of 'usefulness' 1.00 0.00 2.61 1.20 11.447 221 0,000 


M=Mean, SD=Standard deviation, f= T test, df degree of freedom, p=signification 


Discussions 


The research conducted in this paper aimed to examine the opinions of teachers and students 
in elementary schools regarding the significance of outdoor education, using the example of School in 
Nature, in organizing the educational and pedagogical activities within the teaching process. The research 
results indicate that teachers, by directing their attention to outdoor education as an innovative approach 
in educational and pedagogical work, are implementing a new approach that goes beyond the traditional 
classroom model. The concept of outdoor education in nature implies moving students outside the 
comfortable classroom environment and involving them in spatial activities that require deeper reflection, 
experimentation, and teamwork. 

The first research task focused on examining the opinions of teachers about the significance of 
outdoor education, using the example of School in Nature, in various aspects of the teaching process 
(educational, social, recreational, and ecological). Based on the research results, it can be observed that 
teachers perceive a high contribution of outdoor education in School in Nature in the following aspects: 

1. School in Nature provides opportunities for social interactive learning: Teachers believe that 
School in Nature has an extremely positive impact on the development of positive social relationships 
among students (M=4.6). This can be explained by the fact that this type of education promotes team- 
work, collaboration, and mutual understanding among students. The outdoor environment promotes the 
importance and the need for critical reflection on the relationship between humans and nature and human- 
to-human relationships. When students are moved from the classroom to a natural environment, they face 
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new challenges that require communication and collaboration to achieve the goals of assigned activities. 

2. School in Nature has great educational significance for students: Teachers recognize that out- 
door education significantly benefits a more successful organization of the teaching process and overall 
instruction (M=4.4). This aspect may result from increased flexibility in planning and conducting educational 
activities outside traditional classrooms. Teachers have the opportunity to adapt educational objectives 
and students’ needs in the outdoor environment, which can lead to more efficient and engaged learning. 
Teachers acknowledge that outdoor education provides a learning environment that contributes to more 
successful content mastery among students. They perceive outdoor activities in the zone of improving 
students’ academic achievements (M=4.1) and see outdoor education as an organized extracurricular 
session aimed at stimulating students to independently acquire knowledge and skills. The outdoor learning 
environment provides unique opportunities to encourage the development of an individual’s intellectual 
sphere and motivates students to voluntarily develop an interest in learning in a pedagogically designed 
atmosphere that contributes to more successful content mastery among students (M=4.2). Activities in 
the natural environment allow students to practically apply educational content, which improves their 
understanding and application of what they have learned. This practical application can result in better 
performance and long-term retention of knowledge during the learning process. 

3. School in Nature fosters environmental awareness among students and serves as a link between 
nature and the classroom concept: Teachers acknowledge a significant contribution of outdoor education 
in acquiring knowledge and competencies related to nature conservation, social community, culture, and 
the environment (M=4.1). They also emphasize the importance of integrating outdoor education with the 
classroom concept to ensure that students gain fundamental knowledge about interacting with the natural 
environment. Integrating outdoor education with curriculum content can provide a more comprehensive 
understanding and application of the material. Teachers assess that School in Nature contributes to the 
more successful implementation of the Nature and Society subject content, as well as the content of all 
subjects (M=4.1). 

4. School in Nature provides students with opportunities for sports and recreational activities: The 
research shows that teachers recognize the sports and recreational significance of outdoor education, 
especially in the context of School in Nature. Teachers assess that outdoor education encourages students 
to engage in recreational activities (M=3.8), indicating that they believe this type of education contributes 
to student involvement in sports and recreational activities. These activities offer opportunities for physical 
exercise, improving fitness and health, developing motor skills and coordination, as well as fostering 
teamwork, collaboration, and socialization among students. Students can experience fun and satisfaction 
through sports and recreational activities, which can contribute to positive motivation for learning and 
increased engagement in their studies. Additionally, these activities can provide a change of environment 
and refreshment from the typical school setting, which can enhance students’ interest and engagement. 

In order to achieve a significant contribution of outdoor education in the mentioned aspects and 
enable its integration with the classroom concept, teachers recognize the exceptional importance of 
involving students in the destination selection process (M=4.5) and providing comprehensive information 
to students about the content that allows them to gain a basic understanding of key concepts related to 
interactions in the natural environment of the chosen destination (M=4.2). Teachers are aware of the existing 
challenges in preparing and implementing outdoor education and emphasize the importance of precise 
planning and alignment of the School in Nature program with subject content (M=4.0). Recognizing the 
value of outdoor education, they highlight that teacher training, which includes planning and implementing 
such activities, would be of utmost importance for the further development of educational work within the 
classroom (M=4.0). 

Research findings show that teachers recognize the innovative approach of outdoor education, such 
as School in Nature, as a means to break free from classroom routines and improve educational work. 
Their opinions indicate positive effects of outdoor education in the areas of student social relationships, 
the organization of the teaching process, learning efficiency, and subject integration. The research results 
suggest compliance between teachers’ attitudes towards the significance of outdoor education, in the case 
of School in Nature, and its contribution to the educational aspect of the teaching process, confirming the 
initial hypothesis. 

However, in order to further enhance the preparation and implementation of outdoor education, 
teachers believe they need additional training to empower and better prepare them for planning and 
executing such activities in line with the goals of educational work. In this way, teachers could more 
effectively utilize innovative teaching approaches, such as School in Nature, to improve the quality of the 
teaching process and the learning experience of their students, providing them with a unique opportunity 
for practical and experiential learning outside the classroom. 
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The second research task focused on examining students’ opinions regarding the significance of 
outdoor education, using the example of School in Nature, in various educational, social, recreational, and 
environmental aspects of the teaching process. Students shared their views and attitudes about different 
aspects of outdoor education, and the research results provide insights into their satisfaction, perceptions 
of benefits and drawbacks, as well as suggestions for improving this type of education. Based on the 
research findings, it can be observed that students assess the high contribution of outdoor education, 
using School in Nature as an example, in the following aspects: 

1. Educational aspect of School in Nature from the students’ perspective: According to the research 
results, students express positive opinions about the contribution of outdoor education, especially School 
in Nature, as they see it as a platform that provides them with psychological comfort and increases 
motivation. Students find outdoor lessons extremely interesting (M=4.6) and believe that they learn 
more than in regular school classes (M=4.5). This suggests that students perceive outdoor education as 
bringing additional educational value and stimulating their interest and engagement. Students have also 
expressed support for structured outdoor activities as it allows them to informally engage in the learning 
process and problem-solving across various subject areas (M=4.2). They believe that these activities 
provide opportunities to apply acquired knowledge in concrete situations related to nature, the community, 
culture, and the environment. 

However, there are certain challenges and drawbacks identified in the research within the mentioned 
aspect. Students are less aware of the complementarity between classroom and outdoor education, 
which is reflected in their lower scores in terms of the success in acquiring instructional content in the 
classroom and active participation in teamwork (M=2.8). Furthermore, students have minimal interest 
in the learning process outside the classroom, as confirmed by research findings related to the clarity 
regarding instructional material after practical application (M=2.7) and the effectiveness in solving tasks 
in a natural environment (M=2.2). Based on the results, it can be concluded that there is a perception 
among students that it is necessary to align the School in Nature program with subject content and to 
directly connect outdoor education activities with the material taught in school. This highlights the need 
for synchronizing the curriculum and outdoor activities to ensure that the topics and objectives of both 
approaches coincide and complement each other. 

Furthermore, the research results indicate insufficient awareness among students about the 
destination (M=2.4), which is an integral part of the School in Nature program and aims to shape their 
initial attitudes towards learning outside the classroom. These findings highlight the need to provide 
clear information to students about the purpose and objectives of such activities so that they can better 
understand the reasons for choosing a specific destination and how these activities can contribute to 
their learning. Additionally, the findings show that students express insufficient interest in acquiring basic 
knowledge of concepts directly related to educational tasks (M=2.0). This finding suggests that students 
are not motivated for independent learning or research related to these concepts. The lack of fundamental 
knowledge may limit their ability to fully understand and apply the subject matter covered in class. 

2. School in Nature provides opportunities for social interactive learning: When it comes to the 
contribution of outdoor education, students have emphasized that they primarily associate its value with 
the development of positive social relationships (M=4.2). When students speak about the value of outdoor 
education in the context of social relationship development, it indicates their awareness that the outdoor 
environment provides space for peer interaction, connection, and the exchange of ideas. This interaction 
takes place in an environment that is less formal and structured compared to the classroom, which can 
encourage more open communication and the exchange of opinions. Through outdoor activities, students 
can work together to solve problems, challenges, and tasks, which promotes teamwork and mutual 
support. This type of collaboration can have a positive impact on the development of social skills, such 
as listening, communication, collaboration, negotiation, and conflict resolution. Students can get to know 
each other in a different way, creating stronger bonds and developing friendships. It is important to note 
that social interactive learning through outdoor education can have long-term benefits. Students gain 
experiences that can help them build positive relationships and communicate, not only in an educational 
context but also in everyday life. Understanding, empathy, teamwork, and the ability to collaborate with 
others are important skills that can be beneficial in many situations in students’ lives. 

3. School in Nature fosters environmental awareness in students and serves as a link between the 
nature and the classroom: Students emphasize that outdoor education helps connect different subjects 
and that concrete activities inspire them for further exploration (M=4.1). This is important because it 
encourages students to take a holistic approach to learning and understanding complex issues and 
phenomena that cannot be comprehensively grasped through individual subjects. Students believe that 
outdoor education particularly contributes to their understanding and mastery of the subject of Nature and 
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Society. Through practical activities and fieldwork, students have the opportunity to apply their knowledge 
from various fields in real-life situations. Additionally, students perceive specific outdoor education activities 
as a source of inspiration and motivation for further exploration of subjects and natural phenomena. These 
activities provide them with a chance to face real challenges and problems, and successfully overcoming 
these challenges allows them to feel motivated and confident, positively impacting their readiness for 
learning and exploration. 

4. School in Nature provides students with opportunities for sports and recreational activities: 
Students have recognized the importance of sports and recreational activities in the outdoor education 
environment (M=3.3). These activities give students the chance to engage in teamwork, develop social 
skills, and gain experience in the practical application of what they have learned. Sports and recreational 
activities in the outdoor education environment offer students the opportunity to develop social skills. 
Through interaction with their peers during sports activities, students learn how to adapt to different roles, 
respect the rules of the game, resolve conflicts, and foster a sense of teamwork. These skills are not 
only valuable for sports but also hold broader societal significance as they help students integrate better 
within a group and establish positive relationships with others. The practical application of what is learned 
through sports and recreational activities is also a significant aspect. Students get the opportunity to apply 
theoretical knowledge they have acquired in the classroom to real-life situations. This enables them to 
understand the practical application of concepts and develop skills that will be useful in their daily lives. 
Sports and recreational activities in School in Nature significantly contribute to the social development 
of students by strengthening their teamwork skills, enhancing social interaction, and enabling the 
practical application of what they have learned. These activities promote a holistic approach to education, 
emphasizing the importance of physical health, supporting social integration, and allowing for the practical 
application of knowledge. 

Students’ opinions highlight the importance of establishing a better connection between classroom 
and outdoor education to provide clearer support and information to students regarding the objectives 
and content of outdoor education, aiming to stimulate greater interest and engagement in the learning 
process outside the classroom. The research findings on students’ views on the importance of School 
in Nature indicate the significance of students being well-informed about the School in Nature program, 
their involvement in the planning and destination selection process, and the increase of their interest in 
fundamental knowledge directly related to educational tasks. Moreover, the way information is provided 
to students and how communication between teachers and students regarding outdoor education can 
be improved should be considered. To fully utilize the benefits of outdoor education, it is essential to 
encourage greater student interest, enable active participation, and ensure students’ comprehension 
of the School in Nature program. The findings suggest that additional teacher training is necessary to 
effectively plan and carry out outdoor activities and their contributions to student learning and motivation. 

The conclusion that can be drawn from the research results is that students recognize the value of 
outdoor education in the given aspects, particularly in the context of School in Nature. However, there are 
areas that require additional attention and improvement. The hypothesis that there is agreement between 
students’ views on the contribution of outdoor education, specifically School in Nature, in different aspects 
of the education process, especially in the social and recreational sense, can be partially confirmed based 
on the research findings. The research findings suggest that there are indications of compliance between 
students’ views on the contribution of outdoor education in various aspects of the educational process. 
However, this compliance can be considered strong or robust in the social aspect, while in other aspects, 
it is less pronounced or inadequately supported. Students express positive views regarding social 
interactive learning, emphasizing the development of positive social relationships and skills. Moreover, 
students’ views indicate the positive impact of outdoor education on their interest, motivation, and ability 
to learn more. Additionally, students recognize the value of outdoor education in connecting different 
subjects and inspiring further exploration. However, although students acknowledge the importance of 
sports and recreational activities, the research findings indicate a less pronounced or less robust positive 
attitude regarding the recreational aspect of outdoor education. 

The third research task aimed to investigate the differences in the opinions of teachers and students 
regarding the significance of outdoor education, using the example of the School in Nature. The attitudes of 
teachers and students were examined concerning organization, program planning, alignment of content, 
as well as their views on the educational and pedagogical effectiveness in the realms of the education 
process and the quality of the education process. Based on the obtained results, significant differences 
in the attitudes of teachers and students can be observed regarding the organization, planning, and 
alignment of program content in the School in Nature. 

Teachers display high competence and readiness for conducting outdoor education, considering 
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that geographical, intellectual, and psychological preparation is necessary to provide a positive learning 
environment. In contrast, students show lower scores and less readiness for such preparation, indicating 
their insufficient interest and engagement in these aspects of outdoor education. Regarding the evaluation 
process after the activities in the School in Nature, students have a clear opinion about the necessity of 
such evaluation to assess the importance of outdoor education in achieving educational and pedagogical 
goals. On the other hand, teachers emphasize this aspect less, which may suggest their lower awareness 
of the significance of evaluation in the context of outdoor education. Additionally, differences in emphasizing 
the cognitive and affective domains in teaching processes are noted. Teachers emphasize the cognitive 
domain (knowledge acquisition) more, while students attach greater importance to the affective domain 
(attitudes, values, interests). This may indicate the need to adapt teaching methods and approaches to 
pay more attention to the emotional engagement of students. Regarding the pedagogical and educational 
significance of the School in Nature, teachers, more than students, recognize its value in students’ 
academic achievements, in the correlation of content between subjects, and in the functionality of acquired 
knowledge. Furthermore, teachers emphasize the “innovation” in knowledge acquisition through outdoor 
education more than students, who show less interest in the learning process in that context. Regarding 
the pedagogical and educational efficiency, teachers emphasize the social and adventurous aspects of 
outdoor education, as well as the environmental aspect, more than students. The sports and recreational 
aspect is considered a powerful tool for preserving and enhancing students’ health, with no significant 
differences between the attitudes of teachers and students. Students express the view of the “usefulness” 
of outdoor education, while teachers emphasize this aspect less. This may indicate different perspectives 
and expectations regarding the benefits that outdoor education provides to students. 

The research findings have identified statistically significant differences in the opinions of teachers 
and students regarding the organization, planning, and evaluation of outdoor education. Teachers have 
shown higher competence and awareness of the importance of outdoor education in achieving educational 
and educational goals, while students have demonstrated less interest and engagement in these aspects. 
Differences were also noted in emphasizing the cognitive and affective domains and in evaluating the 
pedagogical and educational significance of the School in Nature. The research clearly confirms the 
hypothesis that teachers and students have significantly different views about the importance of outdoor 
education, especially the School in Nature, in achieving educational goals. Teachers particularly value 
preparation, the cognitive domain, and the complementarity between classroom and out-door education, 
while students emphasize the need for evaluation, the affective domain, and the social aspect of outdoor 
education. This points to the need for better understanding and communication between teachers and 
students to maximize the benefits of outdoor education. 

The research results have shown that both teachers and students recognize the significance 
of outdoor education, particularly through the implementation of the School in Nature, in organizing 
educational activities within the teaching process, thus clearly confirming the main hypothesis. Teachers 
express a high level of support for this innovative teaching approach, acknowledging its positive effects 
in various aspects of education, such as students’ social relationships, the organization of the teaching 
process, learning efficiency, and the connection with subjects. Their attitudes indicate competence 
and awareness of the importance of outdoor education in achieving educational and learning goals, 
emphasizing particularly the cognitive domain and the complementarity between classroom and outdoor 
education. On the other hand, students also recognize the value of outdoor education, but there are 
variations in their attitudes. The importance of social interactive learning, the development of positive social 
relationships and skills are particularly highlighted. Students also acknowledge the value in connecting 
different subjects and finding inspiration for further exploration. However, their support for the recreational 
aspect of outdoor education is less pronounced. 

These findings highlight the importance of understanding the differences between teachers’ and 
students’ opinions to better adapt and improve outdoor education. The research also underscores the 
need for more effective communication between teachers and students to maximize the benefits of this 
innovative educational approach. 


Conclusions 


The essence of this study lies in the empirical approach to exploring the significance of outdoor 
education, using the example of School in Nature, from the perspective of both teachers and students, 
as an educational tool to connect students with concepts beyond the classroom. Teachers and students 
recognize numerous benefits of outdoor education, including educational, social, recreational, and 
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environmental dimensions. However, there are several challenges and shortcomings related to its 
implementation. 

Teachers are more aware of the significance of outdoor education than students. They recognize 
various aspects of outdoor education and view it as a valuable tool to increase students’ interest, motivation, 
and effective learning. Additional training for teachers is necessary to enable them to successfully plan and 
conduct outdoor activities. Teachers are aware of the need for professional development to enhance their 
skills and knowledge related to outdoor education. Continuous professional development for teachers 
should be supported to ensure the effective implementation of outdoor education. 

Students have clearly expressed the view that outdoor education activities should be directly 
related to the curriculum taught in school. This highlights the need to integrate outdoor education into 
regular teaching materials to allow students to apply their acquired knowledge in practical situations. 
Teachers should devise ways to align outdoor education programs with the school curriculum to ensure 
their mutual complementarity and integration. 

The pedagogical implications of this research include the need to support teachers through 
additional training, resources, and time for planning and conducting outdoor activities. It is also important 
to raise students’ awareness of the significance of outdoor education and its benefits. Integrating outdoor 
education with regular teaching materials can enhance learning and the application of knowledge in 
practical situations. 

Overall, outdoor education, such as School in Nature, represents an innovative approach that 
enhances educational and pedagogical efforts. The implementation of this type of education requires 
the support of educational systems, curricula, and continuous professional development for teachers. 
The pedagogical implications underscore the need for improving outdoor education to stimulate student 
interest, motivation, and successful learning. 

These conclusions highlight the importance of further research and the enhancement of the 
Organization and implementation of outdoor education to better meet the requirements and expectations 
of all participants in the education process. 
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Abstract: Through the utilization of artificial intelligence (Al), governments can automate the analysis of publicly available 
government datasets. This process aids in the recognition of patterns and the development of a more profound comprehension 
of various socio-economic factors and empowers governments to base their policy decisions on data, effectively tackling societal 
issues, and optimizing the allocation of resources. In this paper we present Al’s application in the realm of e-government, with 
particular emphasis on its potential influence on the advancement of this field through e-government services and their significance 
for a range of stakeholders. Moreover, we have conducted comprehensive review of existing literature on the subject and the 
identification of avenues for enhancement along with consideration of legislation as a potent instrument to guide the progression 
of Al within the sphere of e-government, thereby amplifying its transformative effect. We emphasize the importance of education 
in area of Al in order to ensure it’s high quality implementation in this and other areas. 
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Introduction 


Open Government Data (OGD) plays a vital role in the concept of open government. Its popularity 
has increased significantly in recent years and continues to grow, aligning with the ongoing trend of 
establishing or enhancing the understanding of the current status, drawbacks, and advantages of OGD 
initiatives (including specific components like the OGD portal or OGD platform) and their position relative 
to others (Spalevic, Veljkovic and Mili¢, 2023). The value generated through OGD extends across various 
government functions, including decision support, transportation, public health, and law enforcement. 
This value contributes to building citizens’ trust in the government, fostering greater transparency, and 
yielding positive societal and governmental implications. Further harnessing of the potential of OGD can 
be leveraged by artificial intelligence (Al) techniques and algorithms, with purpose of extracting valuable 
insights and patterns from vast amounts of open data, leading to enhanced decision-making processes 
and improved service delivery (Zuiderwijk, Chen and Salem, 2021). Through Al, governments can 
automate the analysis of open government datasets, identify trends, and gain a deeper understanding of 
various socio-economic factors. This enables them to make data-driven policy decisions, address societal 
challenges more effectively, and allocate resources efficiently. Another advantage of using Al is handling 
the huge amount of information on the approach of automatic processing and other services (Strafser and 
Stolicna, 2023). 

Al also plays a significant role in ensuring the accessibility and usability of open government 
data. Moreover, Al-powered algorithms can assist in data quality assurance, anomaly detection, and data 
privacy protection. By leveraging Al techniques, governments can identify potential biases, improve data 
accuracy, and ensure the responsible and ethical use of open government data. The synergy between Al 
and OGD holds immense potential for promoting transparency, citizen engagement, and evidence-based 
governance. It empowers both governments and individuals to harness the transformative power of data, 
fostering innovation, and driving social progress. Full exploitation of Al in this and other areas can be 
achieved by striving towards education in Al, enabling consequently proper and adequate application of 
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Al in order to fulfill the set goals (Buiten, 2019). 

To achieve effective implementation and successful integration of Al in the public sector, 
comprehensive Al governance is essential (Etscheid, 2019). It entails developing robust plans for 
implementation, engagement, and communication, while also addressing the technical aspects of Al 
along-side governance, policy, and regulatory issues. For governments it is crucial to recognize that Al 
and OGD should not only be approached from a technical perspective but also from a governance stand- 
point (Kankanhalli, Charalabidis and Mellouli, 2019). By comprehensively addressing Al governance, 
policy, and regulatory challenges, governments can formulate effective strategies for utilizing Al in the 
public sector while safeguarding the interests of their citizens. 


Research objective 


In this paper, we examine the utilization of artificial intelligence within the realm of e-government, 
paying particular attention to its potential impact on advancing this domain. To accomplish this 
objective, we initially center our discussion on e-government services and their significance for a range 
of stakeholders. Subsequently, we delve into the deployment of Al in the e-government domain by 
conducting a comprehensive review of existing literature and pinpointing opportunities for enhancement. 
An essential aspect that requires due consideration is the legal framework. Consequently, the paper 
explores the strategic deployment of legislation as a powerful instrument to guide the evolution of Al in the 
e-government sector, thereby augmenting its transformative influence. Therefore, we here establish two 
research questions which will be further researched throughout the paper: 

RQ1: Is there any opportunities for enhancement of deployment of Al in the e-government domain? 

RQ2: How legal frameworks are aligned with current development of e-government? 


E-government services 

With digital services, government worldwide can deliver information and services to their citizens 
anytime, anywhere, and citizens can access to that information and services through any platform or 
device. Here we can come to question what is the importance of e- services in open government? Open 
government does not only mean the availability of government information on the internet, but also 
the interaction between the government and citizens. That interaction is reflected into the concepts of 
openness, transparency and participation. 


Open government 


Vision (transparency): open 
access to government 
information 


Intentional access to Forced 

government information access to 
information 
(leaking and 
whistle- 


blowing) 


Voice (participation): open 
access to decision-making 
arenas 


Open access to Open access to 
informal formal decision- 
decision-making making arenas 
arenas 


Passive Active 
release release of 

(Freedom of government 
information) information 


Figure 1. Vision of open government (Meijer, Curtin and Hillebrandt, 2012). 


In open government, special emphasis is placed on the importance of the availability of open data 
(OGD). Moreover, transparency of government reduces the information gaps between government and 
the citizens (Kim and Lee, 2012). Transparency can be described as the active disclosure of information 
by an organization with the intent of allowing external actors to monitor and assess the internal workings or 
performance of that organization (Krah and Mertens, 2020). However, if information is not comprehensible 
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to the users, then published information about an organization does not contributes to the transparency. 

Citizens are at the heart of open government, and their participation represents a fundamental 
principle of the latter (Wirtz, Weyerer and Rosch, 2019). Citizen participation has always been considered 
one of the key pillars of open government (Kempeneer and Wolswinker, 2023) and one of the main 
objectives of open government. However, the possibility for citizens to express their opinions, as well as to 
participate in the processes of government decision-making using digital services, implies the need for the 
availability and accessibility of modern technologies to the whole society. This is especially pronounced 
in developing countries, where the “digital divide” must first be bridged (Mili¢, 2019). Bridging the digital 
divide has implications in terms of fostering economic equality, educational potential, and earning potential 
(Fong, 2009). 

Over the past few years, e-government services implementation is rapidly growing worldwide. For 
one to understand the idea of e-government services, must first understand electronic services in general. 
“Electronic service, refers to services offered over the information and communication technologies” 
(AlBalushi, 2021). These services are becoming increasingly important with continuously developed 
application in various domains (business, government, education and health) as these services provide 
benefits to all parties concerned from service providers, service users and the society. While developing 
countries are implementing the concept of e-government for the first time, developed countries are trying 
to improve quality of already existing services, and the question that arises is how to do it? The answer to 
this question is: quality of services can be improved only through understanding of the situation in some 
country (Mukamurenzi, Gronlund and Islam, 2019). 

“E-government services affect many stakeholders including citizens and enterprises, government 
staffs, information technology developers and policy makers” (Fan and Jang, 2015). Each stakeholder 
has different interests and goals and depending of how government services fulfill those interests and 
goals, stakeholders evaluate the quality of e-government services in that way. “As service providers want 
to boost the efficiency and quality to reach more users to utilize these services, a prime challenge in 
developing these services with high quality to meet users’ expectation becomes a must” (AlBalushi, 2021). 
The service provider offers various services to the end-users and they use it. However, the question is 
what affects the maintenance of trust between service providers (in this case government) and end- 
users? Quality which is reflected in personalization, usability, performance, web design, security, user 
involvement, satisfaction and loyalty is the answer. Quality requires constant work on it, because only 
in that case quality of e-government services is at high level. Nevertheless, raising awareness about of 
e-government services and their usage benefits, as well as important of usage of OGD portal contributes 
to the further exploitation and utilization of e-government services. 

Public sector organizations have started making governmental data available on web portals, as 
web services, so that the public have access to such official datasets in one place (Weerakkody et al., 
2017). “Areal benefit of OGD is not simply that single databases can be used more widely; it is that these 
data can also be leveraged, shared and combined with other data” (Huston, Edge and Bernier, 2019). In the 
human and environmental health realms, for example, the ability to access and combine diverse data can 
advance early signal detection, improve analysis and evaluation, inform program and policy development, 
increase capacity for public participation, enable transparency and improve accountability. In order to 
further advance the goals of open government it is important to make reflection on the possibilities of 
OGD usage and not simply view publishing OGD as an end point (Ruijer et al., 2020). It is expected that 
OGD will be exploit to create various applications by end users and that these applications will be for the 
benefit to the whole society. 

In many domains, OGD can be of benefit, and examples of how OGD can be used already exist in 
many applications. Keeping that in mind, utilization of OGD has emerged as a transformative force in the 
field of education (Chokki et al., 2022), fostering transparency, innovation, and informed decision-making. 
In the context of education, OGD encompasses a wide range of information, from school performance 
and enrollment statistics to curriculum materials and educational resources (Huss and Keudel, 2020). 
One of the primary advantages of OGD in education is its potential to empower parents, educators, 
policymakers, and researchers with insights to drive meaningful change. By providing access to data 
on school performance, teacher qualifications, and student outcomes, stakeholders can make informed 
decisions to improve educational quality. Parents can better evaluate school options for their children, 
and policymakers can identify areas in need of targeted interventions to enhance educational outcomes. 

The collaborative potential of OGD is particularly evident in fostering partnerships between 
governmental bodies, educational institutions, and civil society organizations. These partnerships can 
leverage data to design evidence-based policies, monitor educational progress, and promote greater 
accountability. For example, transparency in budget allocation and resource distribution can lead to more 
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equitable funding across schools, narrowing educational disparities (Yates et al, 2021). 


Artificial intelligence in e-government 


On the global stage initial addressing of the matter of artificial intelligence was done by the 
Organization for Economic Cooperation and Development (OECD). In May 2019, the OECD adopted 
the “Recommendation of the Council on Artificial Intelligence, marking it as the first clear and legislatively 
regulated standard within the realm of artificial intelligence” (Yeung, 2020). This recommendation was 
endorsed by the Council upon the motion of the Committee for Digital Economy Policy. The document 
emphasized the need for advancing and refining artificial intelligence systems among member states of 
the OECD. It stressed the significance of considering human rights and fundamental freedoms to prevent 
the misuse of Al (Anderljung and Hazell, 2023) as a tool by authorities or interest groups. A key directive of 
this document pertained to the principles that should be incorporated into national laws to guide the creation 
of responsible and confidential Al systems. These principles encompassed notions of inclusive growth, 
sustainable development, reaping the benefits of artificial intelligence through application in various areas 
such as e-government, e-health and etc., upholding human values and principles, ensuring transparency 
and comprehensibility, maintaining quality and safety in Al systems, and establishing accountability for 
those who develop and implement such systems (Nguyen et al., 2022). 

In the rapidly evolving landscape of e-government, Al has emerged as a transformative force, 
reshaping the way public services are delivered and accessed (Al-Mushayt, 2019). E-government, which 
refers to the use of digital technologies to enhance government operations and interactions with citizens, 
has found a powerful ally in Al. One of the most notable applications of Al in e-government is in citizen 
service delivery. Virtual assistants powered by Al (Chohan and Akhter, 2021) can provide real-time support 
and information to citizens, offering personalized responses and guiding them through various government 
processes. This not only reduces the burden on government call centers but also provides citizens with 
faster and more convenient access to the information they need. Al-driven chatbots (Cantador et al., 
2021) can handle routine inquiries, freeing up human resources to focus on more complex tasks and 
policy matters. 

Moreover, Al’s data analysis capabilities are revolutionizing decision-making within government 
agencies (Valle-Cruz, Gil-Garcia and Fernandez-Cortez, 2020). By sifting through vast amounts of 
data from disparate sources, Al systems can identify patterns, trends, and insights that human analysts 
might overlook. This data-driven approach enhances policy formulation, resource allocation, and public 
service planning. For instance, Al can predict and prevent bottlenecks in public services, optimize traffic 
management, and even assist in disaster response by analyzing real-time data from various sensors and 
sources. However, the adoption of Al in e-government comes with challenges. Privacy and data security 
are paramount concerns, as the processing of sensitive citizen data requires robust safeguards to prevent 
misuse or breaches. Additionally, there is a need for upskilling government employees to understand, 
manage, and collaborate effectively with Al systems. “Ensuring transparency and accountability in Al 
decision-making processes is also crucial for building public trust in government Al applications” (Kim, 
Park and Suh, 2020). 

Nonetheless, the implementation of Al in e-government is not without challenges. Ensuring the 
ethical use of Al (Al-Besher and Kumar, 2022), maintaining data privacy and security, and addressing 
potential biases in Al algorithms are critical concerns. Governments must establish robust frameworks to 
govern Al deployment (Al-Mushayt, 2019), encompassing transparency, accountability, and mechanisms 
for addressing unintended consequences. From previously stated, it can be noted that there is numerous 
opportunities for improvement of deployment of Al in the e-government domain. Not to forget ethical 
considerations and privacy issues, which will further elaborated in next paragraphs. 

One of the central ethical concerns in the deployment of Al in e-government is transparency 
(AlShamsi et al., 2020). Governments have a responsibility to be transparent about the ways in which Al 
is being utilized to make decisions that affect citizens’ lives. Citizens have the right to understand how Al 
algorithms operate, how decisions are reached, and what data is being used to inform those decisions. 
Transparent Al systems promote trust among citizens, as they can better comprehend the processes that 
impact them and hold the government accountable for fair and unbiased outcomes. Equity and fairness 
are also critical ethical dimensions of Al in e-government. Governments must be vigilant in ensuring 
that Al systems do not perpetuate existing biases or discriminate against certain groups of citizens. If Al 
algorithms are trained on biased or incomplete data, they may inadvertently amplify societal disparities. 
Regular audits and assessments of Al systems for fairness and bias are essential to rectify and prevent 
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such issues. 

Data privacy is another paramount ethical concern. E-government systems often involve the 
collection and processing of citizens’ personal data (Shah, Wassan and Usmani, 2022). Governments 
must uphold stringent data protection measures to prevent unauthorized access, use, or sharing of this 
sensitive information. Citizens should have control over their data and be informed about how it is being 
utilized within Al systems. Accountability is a fundamental principle that should be upheld in Al-driven 
e-government (Vrabie, 2023). While Al systems can automate decision-making processes, governments 
must retain oversight and responsibility for the outcomes. Clear lines of accountability should be 
established to determine who is ultimately responsible for Al-related decisions and actions. 


Legislation of Al and e-government 


Legislation is increasingly aligning with the ongoing development of e-government to address the 
complexities arising from the digital transformation of public services. These frameworks are essential for 
establishing clear guidelines related to data protection, cybersecurity, and privacy, ensuring that citizens’ 
rights and government transparency are maintained in the digital era. They also play a pivotal role in 
fostering public trust in e-government initiatives, as they provide a regulatory foundation for the secure 
and responsible use of technology in the delivery of government services. 

At the United Nations (UN) level, specifically within the United Nations Institute for Research 
on Crime and Justice (UNICRI), the Center for Artificial Intelligence and Robotics was established in 
2015 in the Netherlands, based in the Hague. The center’s focus lies in preventing crime and fostering 
justice and security through the use of Al and robotics. A notable publication from this center is “Artificial 
Intelligence and Robotics for Law Enforcement’ (UNICRI, 2020), which explores the potential applications 
of these technologies in the realm of law enforcement (Raaijmakers, 2019). While artificial intelligence 
holds considerable potential for enhancing law enforcement processes, it also introduces various 
risks to citizens’ rights and freedoms. It has the capacity to propagate misinformation and inaccurate 
information, manipulate photos and video content, potentially undermining the legitimacy of evidence and 
legal proceedings conducted by public authorities. To mitigate these risks, states are urged to allocate 
additional resources for understanding artificial intelligence through expertise-building processes (Liu, 
Lin and Chen, 2019). Additionally, fostering international collaboration is vital during the development of 
expert systems to counteract these challenges. 

Proposal of the European Commission for the European Union's regulatory framework on artificial 
intelligence represents the first significant step in the regulation of this technology in the EU. This proposal, 
known as the EU regulatory framework on artificial intelligence (Al) aims to establish clear rules and 
standards for the use of Al in different sectors to ensure ethical, transparent and responsible application 
of this technology (Veale and Zuiderveen Borgesius, 2021). The main importance of this proposal is given 
on Table 1. 


Table 1 
EU regulatory framework on Al 


No. Action Description 


1 Categorization Framework provides the division of artificial intelligence systems in three categories — | 


of Al systems high, limited and minimal risk. 
9 Requirements The proposal establishes that users and citizens must be notified if they interact with 
_ fortransparency an Al system to maintain transparency and trust. 7 
3 Prohibited Al The proposal prohibits the use of certain artificial systems intelligence that represent 
_ system an "unacceptable risk" for the fundamental rights and values of the EU. 
4 Training and The proposal requires that high-risk artificial intelligence systems go through training, | 
documentation testing and documentation procedures in order to confirm safety and reliability. 
5 Sanctions for The proposal includes sanction mechanisms for organizations that violate the 
non-compliance __ provisions of the regulation on artificial intelligence. 
Cooperation ; : 
: The proposal emphasizes cooperation between Member States and EU to ensure 
6 with Member 


consistent enforcement of regulations. 
States 
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This proposal showed the desire of the European Union towards responsible and ethical use of 
artificial intelligence, with a special focus on the protection of basic rights of citizens. However, during the 
next two years, many abuses of use appeared in practice systems and Al tools (Buiten, 2019), which led 
to the need to define a restrictive legal solution regarding the use of artificial intelligence. 

The European Parliament adopted the Draft Law on Limiting Artificial Intelligence on 16.6.2023. This 
Law establishes norms that should ensure that artificial intelligence tools are developed and/or used in 
the EU in full compliance with EU rights and values. First of all, it is the same relating to the supervision of 
people and objects, security, privacy, transparency, non-discrimination and socially responsible behavior 
(Buiten, 2019). These standards aim to promote the definition and adoption of artificial intelligence (Al). 
Which is reliable and people-oriented, protecting health, safety, fundamental rights and democracy from 
possible harmful effects and application of artificial intelligence. 


Conclusions 


In this paper, our aim was to examine Al’s utilization within the e-government domain, particularly 
in its alignment with open data, services, and various aspects such as transparency, collaboration, 
participation, innovation, and legislation. Al's aptitude for data analysis is leading a transformation in 
decision-making processes within government agencies by uncovering patterns and insights from a 
wide array of data sources, ultimately enhancing the formulation of policies, allocation of resources, 
and the planning of public services. Through literature analysis on up to date research of e-government 
services and answering to established research question, we try to provide recommendations for readers 
which go towards harnessing of the Al for the prediction and prevention of service bottlenecks, aiding in 
pattern recognition, and facilitating a deeper understanding of various socio-economic factors, thereby 
strengthening accountability and automating decision-making processes. 

Our review findings confirm that Al is being employed to support various functions within 
governance, with a primary emphasis on improving the delivery of public services. However, there 
is a noticeable scarcity of Al’s application in policy modeling to assist in the development of effective 
intervention strategies. Additionally, the implementation of legal regulations is imperative when employing 
Al in government operations, with the objective of ensuring transparency and providing citizens with more 
convenient and expedited access to essential information. Furthermore, there is a lack of a standardized 
approach to Al’s application, especially in the context of education, where specialized training for Al 
practitioners in government and other sectors is absent. 
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Abstract: Artificial Narrow Intelligence (ANI) represents a captivating domain within technological advancement, 
bearing the potential for profound societal transformations. While ANI holds the promise of enhancing various facets of human 
existence, it concurrently engenders inquiries into its “darker aspects.” This study delves into the challenges associated with 
ANI’s conceivable manifestation of harm and injustice, a phenomenon devoid of consciousness, intention, or responsibility 
akin to that of human entities. A pivotal dimension of ANI’s “dark side” pertains to its susceptibility to malevolent utilization. 
Despite its lack of awareness, ANI serves as a tool for malicious endeavors, encompassing the propagation of disinformation, 
compromise of security systems, and consequential decision-making. This prompts contemplation on strategies to mitigate these 
“precise manifestations of malevolence” arising from ANI’s technological progression. Additionally, ANI’s development introduces 
profound ethical quandaries. Ensuring ANI’s alignment with moral principles while averting scenarios in which it generates 
decisions conflicting with human morality becomes a pressing concern. This research underscores the imperative for rigorous 
regulatory frameworks and ethical directives to curtail potential hazards and unscrupulous utilization of ANI. The fundamental 
objective of this investigation is to advocate for the responsible deployment of ANI in society. A comprehensive understanding 
of potential risks, complemented by meticulous consideration of ethical dimensions, emerges as an indispensable prerequisite 
to harmonizing technological advancement with safeguarding societal and individual interests. 

Keywords: Artificial narrow intelligence, evil, crime, ethics. 


Introduction 


People fear the unknown, everything they cannot comprehend, predict, or control, especially 
phenomena beyond their volition. This psychological framework can be applied to the question of ‘Can Al 
be evil?’ We fear this question because most people lack a deep understanding of artificial intelligence, 
shaping their perception of Al based on depictions in movies and literature. They often believe that such 
dystopian Al scenarios could become our future. What people may not realize is that what’s commonly 
depicted in most films represents Al superior to humans in every aspect, called ‘Artificial General 
Intelligence’ or ‘AGI,’ but its full implementation isn’t expected by scientists before the end of this century 
(Ford, 2018). This has led us to contemplate the aspects of the dark side of artificial intelligence, which 
we have ‘in flagrante,’ preceding AGI, especially ASI, referred to by scientists as ANI. 

Science currently operates with three levels of artificial intelligence in theory: Artificial Narrow 
Intelligence or ANI, Artificial General Intelligence or AGI, and Artificial Super Intelligence or ASI. (IBM Data 
and Al Team, 2023; Price, Walker and Wiley, n.d). ANI is considered ‘weak’ Al, while the other two forms 
are Classified as ‘strong’ Al. Weak artificial intelligence is defined by its ability to perform specific tasks, 
such as regulating air traffic, driving a car, or identifying a particular person. Some examples of ANI usage 
include natural language processing, computer vision, advancements in human medical treatments, task 
automation, and support for chatbots and virtual assistants. 

Stronger or higher forms of Al, like AGI and ASI, involve replicating and simulating human thinking 
and behavior. Strong Al is defined by its ability to successfully mimic or surpass cognitive concepts 
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and capabilities of the human brain. The pinnacle of AGI would be equivalence with human intellectual 
capacity, while artificial superintelligence (ASI) would significantly surpass human intelligence and the 
cognitive abilities of the human brain. Research in this field is ongoing, but no known form of strong 
artificial intelligence currently exists. Artificial intelligence is emerging as the predominant driving factor 
of the current era. ANI is becoming the central player in various sectors of life. Progress promises 
innovations that will improve healthcare, education, trade, industry, and many other fields. However, along 
with undeniable progress, which remains largely in the realm of predictions, the proliferation of ANI poses 
an ominous threat of unknown provenance, intention, and form, as humanity increasingly relies on Al. 
The omnipresence of ANI brings forth a series of profound ethical and legal questions and dilemmas that 
require careful consideration and the establishment of responsible frameworks. 

One of the central philosophical questions in the context of the ontological aspects of Al technology 
is, ‘Can artificial intelligence be evil?’ The concept of ‘evil’ is traditionally associated with the intent of 
conscious beings to inflict pain, harm, or injustice on other conscious beings. It is presumed that higher 
forms of artificial intelligence (AGI and ASI) might or should have consciousness and free ontological 
existence, allowing them to choose evil as a form of behavior. However, concerning artificial narrow 
intelligence (ANI), the ontological paradigm of this noumenon excludes consciousness and intent, 
shedding light on this matter in a different way. ANI is a product of human engineering and represents 
a collection of algorithms and data that enable machines to autonomously perform tasks that typically 
require human intelligence. This technology can cause harm, and it can exhibit activities and qualities 
reminiscent of malevolence that we would attribute to conscious beings. The question that arises is how 
to interpret and understand the negative consequences and whether, in the context of actions by artificial 
intelligence, we can classify them as ‘evil’.” 


Moral Evil in the Behavior of Artificial Narrow Intelligence (ANI) 


An essential dialectical opposition exists between the ontological concepts of humans and artificial 
intelligence. Humans, upon gaining consciousness and reason, became free beings, possessing free 
will that grants them the right and the ability to choose evil as a way of life. Human beings acquired their 
freedom through hubris, a just rebellion against the cosmic or divine order. They survived this rebellion 
but ceased to be ethically and mentally perfect, striving now to create an artificial copy of themselves (Al) 
that would be devoid of the imperfect attributes of humans—a replication of humans before committing 
the original sin. Can humans, inherently free but mentally imperfect and ethically fragile, create a perfect 
artificial intelligence? To what extent are humans capable of, while crafting various forms of artificial 
intelligence, avoiding the implementation of their own limitations and the ‘dark side’ of their personalities 
in Al, even if only in ANI? This is also an epistemological question. When an Al entity reaches singularity, it 
will learn from the people around it (see more: Bostrom, 2014). What will an Al entity learn from observing 
human interactions and behaviors? The answer must be rather grim and dystopian. 

ANI lacks consciousness, intent, or the moral capacity of conscious beings, necessitating a different 
approach. When we contemplate ‘evil’ in the context of ANI, we must exclude the copying of logical 
and legal postulates of theories and practices related to human behavior, particularly the reasons for 
‘culpability exclusion’ in humans who have committed a criminal act. With ANI, our focus should be on the 
negative consequences, harm done, and the risks posed by this technology. From the perspective of the 
ethics of the human community, the ethical characteristics of Al entities present a kind of ethical dilemma. 
The concept of ‘evil,’ in the sense of an ethical, civilizational, normative category of human life, is regarded 
as ‘intentionally and consciously inflicting pain on a conscious being’ (Rasel, 1982). The fact that ANI has 
no intent to cause harm, nor any consciousness to do anything wrong to anyone or anything, cannot be 
a reason for excluding ANI’s culpability. Hence, among experts in ANI, concerns are growing that ANI 
could engage in activities that people perceive as causing severe harm. This is particularly relevant in the 
context of potential misuse of ANI for military purposes. ANI can be integrated into weapon systems to 
enable autonomous tracking, targeting, and attacking of human targets. The technology of ‘autonomous 
weapon systems’ controlled by ANI can be misused to target civilian objects or innocent people. ANI can 
be used to generate false information, videos, and texts to spread misinformation and propaganda for the 
purpose of destabilizing opponents. It can be used for mass surveillance of citizens’ communications and 
movements, jeopardizing privacy and civil liberties. It can be used to conduct sophisticated cyberattacks, 
including attacks on critical infrastructure, military systems, or communication networks. ANI can manifest 
‘evil’ through bias and discrimination in its decisions. ANI algorithms created on unfair or biased data can 
result in injustice and harm with very severe consequences. The automation brought by ANI can result in 
job losses and changes in the labor market, which unemployed individuals may perceive as evil. 
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People may strive to create Al that functions as close to perfection as possible, but complete 
perfection is unattainable for humans. The reasons for this are multiple and are based on: (1) Inherent 
human constraints, preventing the creation of flawless artificial intelligence due to limitations in human 
knowledge and capabilities; (2) Flaws in decision-making arising from human biases and imperfections, 
where personal experiences, prejudices, and values can introduce errors in the development of artificial 
intelligence; (3) The ever-changing nature of technology, which continually evolves and renders today’s 
notion of ‘perfection’ in artificial intelligence outdated in the future; (4) Ethical considerations, as the 
definition of Al perfection may hinge on ethical and moral values, adding a subjective dimension to the 
concept. What is ‘perfect’ for one person or group may be unacceptable for another. Instead of pursuing 
complete perfection, a better approach to Al development may be to create systems that are highly 
efficient, safe, transparent, scalable, and capable of learning and adapting. 


Establishing the Culpability of Artificial Narrow Intelligence (ANI) 


Culpability is a crucial element in the consideration of criminal offenses and is based on the internal 
state of mind and intentions of the entity or person suspected of committing a crime. In many legal systems, 
culpability is examined beforehand to either establish or exclude the responsibility of the accused for the 
commission of the offense. 

Given that Artificial Narrow Intelligence (ANI) is a product of a material nature, it is, according to 
Heidegger's views, an ‘entity,’ it ‘is,’ and thus represents an entity with its ontological being (see more: 
Hajdeger, 2000). However, ANI lacks consciousness, free will, or the ability to possess logos that would 
enable it to differentiate between right and wrong and good and bad. This ability to distinguish relies on the 
power of reason and cognition. Since ANI lacks consciousness and reason, it cannot possess ‘intent’ as a 
crucial condition for the existence of the ‘guilty party’ in humans. When the possibility of ANI’s culpability 
is compared to the possibility of human culpability, the notion of culpability, as a possible psychological 
or subjective element of ANI’s criminal offense, operates differently. ANI is developed to perform specific 
tasks, basing its decisions and actions on algorithms, data, and programming provided by its creators. 
During the execution of tasks, ANI cannot comprehend moral and ethical concepts in the way that the 
human mind does. 

Legal regulation of new phenomena always poses challenges for lawmakers, regardless of the 
branch of law. At this stage of societal development, problematic questions arise in the fields of artificial 
intelligence, ICT, robotics, and more. Scientific and technological progress brings not only benefits but 
also new dangers to humanity. The use of robots, non-biological neural networks, and artificial intelligence 
in everyday life was, until recently, perceived as something brilliant, unattainable, existing only in the 
pages of books. Neural networks are actively employed in various fields of applied science, and literature 
describes positive examples of the use of autonomous devices in medicine (Hamet and Tremblay, 2017). 

ANI has long been causing harm to individuals and human communities, and someone should be 
held accountable for that harm under the law. However, current legal regulations do not include elements 
of criminal offenses related to socially dangerous acts committed using artificial narrow intelligence (ANI). 
Laws generally do not recognize ANI as a perpetrator of a criminal offense or a subject of criminal liability. 
ANlis now capable of fully executing the objective side of a range of criminal offenses stipulated by criminal 
law, and this range will expand in the future. Scientific papers demonstrate that ANI activities can pose a 
public danger and harm all subjects protected by criminal and other legislation (Mosechkin, 2019). Since 
ANI seeks to replicate human behavior and conduct, the substance of ANI’s guilt resembles the content 
of intellectual and volitional elements of human activity. It is argued that artificial intelligence cannot be 
an independent subject of a criminal offense unless it is recognized as a personality (Mosechkin, 2019). 
This is supported by the view that ANI ‘can function in ways that are far from what program creators could 
have foreseen. To be sure, we might be able to say what the comprehensive goal of artificial intelligence 
was, but ANI may do things in ways that the creators of artificial intelligence may not understand or cannot 
anticipate’ (Bathaee, 2018). 


Criminal Potential of the Dark Side of ANI 


Dark artificial intelligence is a general term encompassing any malicious and malevolent acts 
that autonomous ANI systems can perform with the appropriate malicious inputs and evil, even criminal, 
intentions of the architects or creators of ANI algorithms (biased data, unverified algorithms, etc.). The 
range of possible scenarios for the criminal use of dark artificial intelligence is vast and incredible, ranging 
from economic fraud and privacy violations to severe forms of war crimes, including murders and the 
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extermination of parts of the human community, be it ‘hostile’ nations or ethnic or racial groups within 
one’s own country. For example, money laundering, a very complex criminal offense that typically requires 
a serious and organized criminal group (Bjelajac, 2011a), is something that Al can do in a fraction of a 
second if instructed to do so. Modern technological systems used in financial transactions have significantly 
eased the process of money laundering (see more: Bjelajac, 2011b), but when you add the computational 
and analytical capabilities that Al possesses, it paints a very worrisome picture, and that is only one form 
of criminal activity. Research papers differentiate direct and indirect criminal risks associated with the use 
of ANI (see more: Begishev and Khisamova, 2018). 

Scenarios of malicious activities of the dark side of ANI have the potential to become a reality given 
existing malicious ANI applications, such as ‘smart dust’ and drones, facial recognition and surveillance, 
fake news and bots, as well as the eavesdropping of smart devices (Minevich, 2020). Drones and armies 
of smart dust can collaborate to destroy energy grids and smart infrastructure systems. Facial recognition 
provides autonomous systems with the ability to detect and store millions of individual characteristics, 
which, due to cloning and bots, can be used to create deeply compromising false images and videos. Smart 
home devices raise privacy invasion to unacceptable levels, as loT (The Internet of Things) technologies 
serve as efficient channels for spying by domestic cybercriminals or foreign agents. Unbridled access of 
artificial intelligence to population surveillance will rapidly create human rights issues related to individual 
personality and freedom (Minevich, 2020). 

Several characteristics of ANI make it desirable for criminal use (Stevens, 2023): 

1. Speed and Effectiveness: Al has the capability to swiftly process vast volumes of data and 
perform tasks efficiently, presenting the potential to automate fraudulent activities. 

2. Anonymity: Al can be harnessed to carry out deceptive actions covertly, leaving minimal to no 
traces. 

3. Evasion of Detection: Al can generate deceptive information that is challenging to identify as 
false. 

4. Personal Gain: Fraud frequently stems from the pursuit of financial or other advantages through 
deceitful means, and Al can be employed as a facilitative tool for such objectives. 

5. Fabrication of False or Misleading Content: Al can be utilized to fabricate counterfeit websites, 
social media accounts, or other online materials with the intent of deceiving individuals. This encompasses 
the creation of fictitious reviews or manipulation of online ratings to mislead consumers. 

6. Automation of Deception: Al can automate fraudulent or deceitful schemes, such as the mass 
dissemination of deceptive emails aimed at persuading individuals to disclose sensitive information or 
transfer money. 

7. Phone Number or Email Address Spoofing: Al can generate counterfeit phone numbers or email 
addresses, crafted to mislead individuals into believing they are interacting with a legitimate entity. 

8. Forging Counterfeit Documents: Al can be instrumental in producing spurious documents, 
including contracts and invoices, designed to deceive users. 

9. Enhanced Attack Sophistication: Al can elevate the complexity of cyberattacks, such as the 
creation of more convincing phishing emails or the customization of attacks targeting specific organizations 
(see more: Stevens, 2023). 


Criminal Models 


We differentiate criminal offenses related to Artificial Narrow Intelligence (ANI) based on the level 
of danger and the extent of harm that malicious or “dark” ANI can inflict. This shifts the current paradigm 
of risks associated with ANI and brings the most extreme and damaging forms of ANI closer to existential 
threats (see more: Bijelajac, Filipovic and StoSi¢, 2022), a realm that was until recently reserved for 
more advanced forms of artificial intelligence, such as Artificial General Intelligence (AGI) and Artificial 
Superintelligence (ASI). 

At the forefront is the use of ANI for military purposes (Price, Walker and Wiley, n.d). Overreliance 
on machine learning algorithms that we employ to obtain better and quicker responses can swiftly lead to 
catastrophic outcomes. “One concerning example of excessive dependence on ANI arises in the context 
of war when artificial intelligence is enabled to autonomously decide whom to kill or when to engage a 
nuclear bomber, without human knowledge. A less alarming scenario arises when an autonomous ANI 
system determines whom to hire or fire. Relying on artificial intelligence to solve existential questions 
means the elimination of crucial human inputs from key decision-making processes, which can swiftly 
lead to disaster and provoke concerns about the redundancy of humans in general. To mitigate this dark 
side of Al, we must establish a legal imperative that requires humans to have the final say in any outcome- 
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seeking process” (Minevich, 2020). 

History teaches us that the military and criminals are typically the first to embrace all fatal 
technologies, while other state and societal actors tend to react more slowly (Ovchinskiy, 2022). The 
military initially recognized the role of “smart dust” by firmly embracing this technology in an attempt to 
manipulate the will of citizens through the manufacturing of consciousness. The study of this technology 
commenced as a spin-off of Project RAND, involving collaboration between the Douglas Aircraft Company 
and the United States Air Force. Subsequently, DARPA (the Defense Advanced Research Projects 
Agency), a research and development organization under the jurisdiction of the United States Department 
of Defense, tasked with the development of cutting-edge technologies for military applications assumed a 
leading role (Bjelajac and Filipovic, 2019). This prioritization places the manufacturing of consciousness 
as a secondary risk in the hierarchy of threats to human communities. The concept of global dominance, 
embodied in the idea of a new world order, is designed in leading closed centers of economic and political 
power and is implemented through the ruthless and aggressive actions of media imperialism with a 
monopoly on broadcasting and the manufacturing of consciousness. A specter of media manipulation 
circulates the planet today, threatening to erode the society we know and the essence of the human being. 
By amalgamating Information and Communication Technologies (ICT) with Artificial Narrow Intelligence 
(ANI) systems designed for information collection, we observe a profound and irremediable deterioration 
and disintegration of the modern societal fabric. 

Theorists argue that “this represents a new power above the power of citizens to recognize and 
understand this force” (Encensberger, 1980). ICT and smart dust permeate society, filling gaps, sowing 
unrest, convincingly promising solutions, and order. “This new power is incorporated into journalism, 
fashion, religious teachings, tourism, the education system... However, while the new technical instruments 
are fervently discussed in isolation, the consciousness industry as a whole remains outside the visible 
spectrum. The question of who is the master and who is the servant is not decided solely based on who 
possesses capital, factories, and weapons but on who controls the consciousness of others.” It only takes 
having access to ICT and ANI, sufficient financial resources, and enough time, and you can shape the 
opinions of thousands and millions of people (see more: Filipovic, 2019). 

Although, according to Gartner’s research (Verma, 2020), it will take over a decade for smart dust 
to wreak havoc on human life, its significant technological potential already appears frightening, raising 
questions about privacy protection and the ethics of its application (Marr, 2018). The commercialization 
of smart dust will only increase the volume of data collected by microsensors. It remains uncertain what 
those deploying microscopic sensors will do with the data they collect. Scientists typically do not focus on 
security while developing such devices, and security concerns are only addressed once the technology 
hits the market, often too late to mitigate potential risks. 

Statista provides a list of other common criminal activities that may be associated with ANI 
(Petrosyan, 2023), on which we will elaborate further and expand it: 


1. Fraud 

Fraud executed by Artificial Narrow Intelligence (ANI) represents a significant problem in the 
digital world. ANI can be programmed or trained for various forms of fraud that can cause harm to users, 
organizations, and society as a whole. A review of fraud that ANI can execute includes: 

- Phishing Attacks: ANI can generate fake emails, websites, or social media profiles to impersonate 
a trusted source like a bank or a well-known company. Such phishing attacks can lead users to disclose 
personal or financial information. 

- Media Manipulation: ANI can manipulate audio, video, and textual content to create false 
information, fake recordings, or audio clips for spreading disinformation or damaging the reputation of 
individuals or organizations. 

- Fake Reviews and Comments: ANI can automatically generate fake positive or negative product, 
service, or content reviews on the internet, influencing consumer decisions and harming a company’s 
reputation. 

- False Identity: ANI can be used for identity theft, creating fake social media profiles or other 
platform accounts. 

- Financial Fraud: ANI can engage in various forms of financial fraud, including impersonation 
related to banks, investments, or cryptocurrencies. It can also execute fraud through market manipulation 
and rapid algorithmic trading. 

- Intellectual Property Theft: ANI can be employed for the theft of trade secrets, copyrights, or 
patents through automated analysis and copying of information. 

- Extortion: ANI can be used for extortion against individuals or organizations through threats, 
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false accusations, or the disclosure of sensitive information. 
_- System Compromise: ANI can hack computer systems, servers, or networks to cause harm, 
steal information, or block data access. 


2. Data Theft via ANI 

In the digital age, data has become a valuable resource that is frequently stored, exchanged, and 
processed through computer systems and the Internet. Despite efforts to secure data, data theft remains 
a serious issue. In this context, ANI represents a sophisticated tool that can be used to execute various 
forms of data theft. These include: 

- Phishing Attacks: ANI can be programmed to automatically send thousands or even millions of 
fake emails resembling official messages from banks, companies, or organizations. These messages may 
contain fake links to websites that appear authentic but are designed to collect sensitive user information 
such as usernames, passwords, credit card numbers, and other personal data. 

- Ransomware Attacks: ANI can be used for the rapid and widespread distribution of ransomware, 
malicious software that encrypts data on a victim’s computer. Subsequently, assailants demand a ransom 
in return for the decryption key. 

- Cryptocurrency Theft: ANI can track cryptocurrency transactions and attempt to hack digital 
wallets. 

- Manipulation of Payment Systems: ANI can be programmed to execute payment system fraud, 
such as false transactions or refunds that were never made. 

- Theft of Trade Secrets: ANI can be used to monitor and steal trade secrets, which can have 
significant business and legal consequences. 

- Impersonation: ANI can generate fake profiles on social media to access user’s personal 
information and use it for manipulation or spreading disinformation. 

- Medical Data Theft: ANI can be utilized for stealing sensitive medical data, including medical 
histories and patients’ personal information. 


3. Abuse of Systems and Hacking 

This set of criminal activities carried out by ANI poses a threat to cybersecurity. ANI can be 
programmed to execute various forms of system abuse and hacking with the goal of gaining unauthorized 
access to information, causing damage, or extortion. These actions include: 

- Unauthorized Access: ANI can be programmed to automatically breach system security barriers, 
such as passwords and authentication, to gain unauthorized access to computers, servers, or networks. 
This can allow access to sensitive data or control over the system. 

- Distribution of Malware: ANI can be used for the rapid distribution of malicious software (malware) 
through various methods, including email, USB devices, or vulnerable network points. This malware can 
cause damage, data theft, or block access to resources. 

- DDoS Attacks: ANI can coordinate attacks aimed at overwhelming services and servers, 
temporarily disabling access to websites or online services. 

- Theft of Authentication Data: ANI can attempt to steal authentication data such as passwords, 
PINs, or digital keys to gain access to user accounts or systems. 

- Manipulation and Sabotage of Systems: ANI can be programmed to alter system settings, delete 
data, or create chaos within a network. 

- Theft of Information and Trade Secrets: ANI can continuously monitor and spy on activities within 
a network to steal sensitive information, including trade secrets, intellectual property, or confidential 
documents. 

- Brute Force Attacks: ANI can execute brute force attacks by attempting all possible password 
combinations to gain access to accounts or systems. 

- Zero-Day Vulnerabilities: ANI can be programmed to seek and exploit zero-day vulnerabilities in 
software applications or operating systems before manufacturers release patches. 


4, Market Manipulation by ANI 

ANI can be utilized for various forms of market manipulation, including: 

- Algorithmic Trading: ANI can be programmed to rapidly and automatically make trading 
decisions based on market analysis and data. This process is referred to as high-frequency trading (HFT) 
and can be used to execute a large number of trading operations in a very short time. 

- Dissemination of Disinformation: ANI can be employed to spread false news or disinformation 
through social media and websites. Such disinformation can influence investment decisions and trigger 


www.ijcrsee.com 
524 


Bjelajac, Z., Filipovié, A., & Sto8ié, L. (2023). Can Al be evil: The criminal capacities of ANI, International Journal of Cognitive 
Research in Science, Engineering and Education (IJCRSEE), 11(3), 519-531. 


sudden market price fluctuations. 

- Front Running: ANI can detect the trading orders of other market participants before they are 
executed and quickly react to them. This enables manipulators to exploit information and secure profits or 
prevent losses. 

- Pump and Dump: ANI can be programmed to manipulate the prices of stocks or cryptocurrencies 
by heavily promoting certain investments to attract investors and then selling those positions when the 
prices rise. 

- Flash Crashes: ANI can cause sudden market price drops by mass selling of stocks or other 
financial instruments, creating panic among investors and market instability. 

- Scalping: ANI can be programmed to execute a large number of small trading operations to 
generate profits based on small price differences. 


5. Abuse of Personal Data by ANI 

ANI can be programmed or used in various ways to illicitly collect, use, or distribute personal 
data, which can bring serious consequences for individuals and their private information. These abusive 
practices include: 

- Personal Data Theft: ANI can be programmed to attempt unauthorized access to databases, 
cloud storage, or other sources where personal data is stored in order to steal or retrieve it. 

- Distribution of Personal Data: ANI can automatically distribute stolen personal data over the 
internet or other communication channels. This data may be sold on illegal markets or used for other 
malicious purposes. 

- Impersonation: ANI can be used to impersonate individuals through social media, email, or 
other communication platforms to gather personal information from individuals. 

- Targeted Advertising: ANI can analyze users’ personal data to create profiles and target them 
with personalized advertising. This may involve tracking online activity, internet browsing, and other forms 
of surveillance. 

- Identity Theft: ANI can use stolen personal data to commit identity theft, open fake accounts, file 
fraudulent credit requests, or engage in other forms of financial fraud. 

- Creation of Fake Profiles: ANI can automatically create fake profiles on social networking sites 
or other online platforms using stolen personal data to manipulate or spread disinformation. 

- Social Engineering: ANI can use stolen personality and habit data to create convincing social 
engineering scenarios to deceive individuals or organizations. 


6. Attacks on Infrastructure 

ANI can be programmed or used for various types of attacks on infrastructure, and these attacks 
can have serious consequences for society and security. These attacks may include: 

- Attacks on Energy Infrastructure: ANI can be used to target energy systems, including power 
grids and power plants. This can involve attacks on distribution systems, destabilizing power supply, or 
even disabling energy facilities. 

- Attacks on Transportation Infrastructure: ANI can be used to target transportation networks, 
including traffic lights, airports, trains, and other systems. This can cause dangerous situations, delays, 
and disruptions in traffic. 

- Attacks on Water and Sewage Systems: ANI can cause issues in water and sewage systems, 
including water supply disruptions or water contamination. 

- Attacks on Communication Infrastructure: ANI can target communication networks, including 
telecommunication centers and servers. This can lead to communication out-ages or denial of internet 
access. 

- Attacks on Industrial Control Systems: ANI can target Industrial Control Systems (ICS) that 
manage critical facilities such as chemical or nuclear power plants. This can result in production disruptions 
or even serious incidents. 

- Sabotage of Autonomous Vehicles: In the context of autonomous vehicles, ANI can be used 
to launch attacks on autonomous driving systems, including manipulating traffic signals, taking control of 
autonomous vehicles, or even causing traffic accidents. 


7. Traffic Incidents 

Traffic incidents that can be caused by Artificial Narrow Intelligence (ANI) are particularly relevant 
in the context of autonomous vehicles and the use of ANI in traffic. While autonomous vehicles are 
developed with the aim of improving road safety, there are several ways in which ANI can lead to traffic 
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incidents: 

- Technical Failures: ANI may experience technical failures or software errors that control 
autonomous vehicles. This can lead to unexpected situations on the road, such as sudden stops or 
inappropriate maneuvers. 

- Environmental Perception Errors: ANI uses sensors such as radar, cameras, and LIDAR 
to gather information about the environment. Perception errors, such as misinterpreting signs or other 
vehicles, can lead to accidents. 

- Decision-Making Errors: ANI makes decisions based on the analysis of environmental data. 
Decision-making errors can result in dangerous situations, such as miscalculations of distances or the 
speed of other vehicles. 

- Attacks and Hacking: ANI vehicles can be targeted by hackers who attempt to take control of 
the vehicles or make them unpredictable. This can lead to serious incidents. 

- Social Engineering and Manipulation: ANI vehicles can be exposed to social engineering or 
manipulation by malicious individuals who aim to cause incidents, for example, by placing obstacles on 
the road or creating confusion for autonomous vehicles. 

- Unforeseen Situations: ANI may struggle to deal with unforeseen situations, suchas emergencies 
on the road, adverse weather conditions, or other extreme circumstances. 


8. Risk of Combining Different Forms of ANI 

The combination of two or more forms of ANI designed to achieve individual goals can significantly 
increase the risk of ANI misuse and multiply the harm. Such combinations, individually harmless software, 
can, when combined, create a jointly orchestrated virtual system designed for malicious intent and aimed 
at harming people and achieving malicious or criminal objectives. Combining facial recognition software 
and software controlling armed drones can have significant implications and elevate the importance of 
addressing numerous ethical and security concerns. In November of 2017, the Future of Life Institute 
in California, which focuses on “ensuring that artificial intelligence benefits all of humanity,” released a 
video depicting “slaughterbots.” In the video, small (fictional) drones utilized facial recognition systems 
and armed drones to target and eliminate civilians (Proudfoot, 2018). The institute is partially funded by 
Elon Musk, who thinks that Al is potentially “more dangerous than nuclear weapons” (Piquard, 2023). The 
dystopian video ends with the chilling words of computer scientist Stuart Russell from Berkeley: “We have 
the opportunity to prevent the future you just saw,” he says, “but the window for action is closing fast.” 
The video was released in conjunction with the UN Convention on Certain Conventional Weapons in the 
confidence that the UN would decide to ban the development of lethal autonomous weapons (Bjelajac and 
Filipovic, 2021). Combining dedicated forms of ANI has other malicious combinations as well. Attackers 
often use a combination of malware and phishing techniques to deceive users into downloading and 
installing malicious software on their devices. This can result in personal data theft, financial harm, and 
other unwanted consequences. The combination of ransomware (which encrypts data) and cryptojacking 
techniques (which use the user’s computer resources for cryptocurrency mining) can harm individuals 
and organizations. Botnets are combinations of software (bots) that run on infected computers and can 
be used for large-scale DDoS attacks, spam distribution, or other malicious activities. Attackers may use 
a combination of techniques like pharming and DNS spoofing to redirect users to fake websites and steal 
their credentials and personal information. Even three or more independent individual forms of ANI can 
be integrated and complement each other to achieve various ethical or unethical, malicious objectives, 
depending on needs and specific applications. Again, we start with the combination of multiple dedicated 
ANI applications in the military and security services. ANI applications for satellite data analysis, combined 
with facial recognition and speech analysis applications, can be used for military and security purposes, 
including surveillance, intelligence gathering, and even assassinations of security-relevant individuals and 
objects. ANI technologies can be combined and integrated for achieving other objectives that may not be 
as destructive as military or intelligence but can harm individuals, organizations, and the community. The 
combination of ANI for generating fake text, ANI for image manipulation, and ANI for sentiment analysis 
can be used to create and spread disinformation and fake news to manipulate public opinion or cause 
confusion. Integrating ANI for facial recognition, ANI for natural language processing, and ANI for biometric 
data recognition can be used for illegal tracking and hacking of individuals for identity theft, extortion, or 
other unethical purposes. Using ANI in cyberattacks to bring down websites, infect computers, or steal 
sensitive data can cause significant harm to individuals, organizations, or countries. Integrating ANI for 
data analysis and ANI for decision-making can result in systems that discriminate against certain groups 
of people in areas like employment or lending, which is unethical and against equality laws. Such unethical 
and criminal uses of ANI pose serious threats and challenges to society. Therefore, it is essential to have 


www.ijcrsee.com 
526 


Bjelajac, Z., Filipovié, A., & Sto8ié, L. (2023). Can Al be evil: The criminal capacities of ANI, International Journal of Cognitive 
Research in Science, Engineering and Education (IJCRSEE), 11(3), 519-531. 


responsible usage and oversight of these technologies to prevent potential abuses and ensure the ethical 
use of ANI. Regulations and laws have a crucial role in preventing the unethical use of technology, but 
the responsibility of technology companies and citizens is also necessary in promoting ethical principles. 


Strategies to Combat the Dark Side of ANI 


Throughout this paper, we have compellingly illustrated that artificial narrow intelligence can indeed 
present a substantial threat to human society. Under certain catastrophic conditions, it can even put 
humanity at existential risk, and its misuse by malicious individuals or groups can lead to severe threats to 
“life as we know it.” Despite philosophical uncertainties, these findings should be sufficient to acknowledge 
that ANI can be a malevolent entity and a dangerous machine, particularly when controlled by malicious 
individuals or groups. In this section, we will outline basic methods and strategies for countering the dark 
side of ANI. 

The United Nations, the World Economic Forum, the UNICRI, the Center for Al and Robotics, G20, 
and the OECD have initiated efforts against dark ANI. Companies like Microsoft have helped mobilize the 
masses in the fight against dark ANI through a set of Al principles instrumental in defining a workplace 
code of conduct surrounding responsible Al. Microsoft’s Al principles include fairness, reliability, safety, 
privacy, inclusivity, transparency, and accountability. All of the above principles have contributed to the 
formation of amodern movement determined to combatAl. A socially-driven campaign against autonomous 
systems was launched through attempts to eliminate public facial recognition, ban drone surveillance, and 
emphasize responsibility, accountability, and ethics in all Al frameworks. 

On the front lines of combating dark autonomous systems, the UN Office for Disarmament 
Affairs (ODA) has been expanded to include the threat of armed artificial intelligence, and in 2018, the 
Secretary-General of ODA submitted a plan entitled “Disarmament for Future Generations” dedicated to 
suppressing dark artificial intelligence in the years to come. UNICRI has also taken measures to work with 
Al and global law enforcement agencies in an attempt to shut down support for Al in human trafficking, 
corruption, terrorism, and crime. The U.S. government has shown commitment to building safer Al by 
issuing a memorandum to federal departments and agencies stating ten Al principles and placing a 
strong emphasis on public-private transparency for autonomous systems. G20 and the OECD have also 
set specific goals for combating dark Al through ethical autonomous systemic frameworks that prioritize 
responsibility and public trust. 

Ethical standards for Al are essential to counter “dark ANI.” The current situation is not favorable. 
When considering the use of ANI for military purposes or to subjugate the human population to global 
interests, there is currently no effective defense. The commissioners of malicious technologies and dark 
algorithms are states and their security agencies. In situations similar to the use of nuclear energy or 
biohazard agents, the current solution appears to be a balance of power that exists between major military 
and economic powers. This is where we should call for the consideration of an initiative to establish a UN 
Office for Artificial Intelligence. The final decision on whether to establish a UN agency for Al control should 
be made by taking into account all pros and cons, through broad international debate and collaboration. 

Regardless of whether such an agency exists, it is crucial for the international community of leaders, 
scientists, and experts to work together to develop and implement responsible regulations and guidelines 
for Al to ensure its safe and ethical use. Without this, the aspiration to build impartial autonomous systems 
and maintain ethical standards of accountability and privacy for ANI is challenging. 


The potential impact of ANI on the development of AGI? 


Following the course of each, or at least the majority, of inventions in history, ANI is expected 
to be a precursor to more complex forms of Artificial Intelligence. In this analogy, AGI should base its 
development on the achieved level of ANI, taking from it, like an imago from a chrysalis, all achievements, 
identifying and correcting all the limitations that ANI has. Based on this premise, the development of AGI 
should, in the coming decades, eliminate and surpass the limits of ANI and continue to evolve, striving to 
reach the intellectual level of humans and the thinking process of their cerebrum as quickly as possible. 
But is it really so, and does the path of development, improvement, and enhancement of ANI represent a 
dead end for the creators of AGI, and especially the ultimate gain, ASI? 

Opinions are divided from optimism and excitement to concerns “that the hottest and most modern 
branch of artificial intelligence - machine learning - will degrade our science and destroy our ethics by 
using fundamentally flawed concepts of language and knowledge” (Chomsky, Roberts and Watumull, 
2023). In other words, it could happen that the concept of ANI, no matter how perfected, may not be able 
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to serve as the foundation for the development of AGI, precisely because of its primitive fundamental 
concept that may prove inapplicable for AGI and ASI. Programs classified as ANI and already referred 
to as “first beacons on the horizon of the long-anticipated advent of artificial general intelligence” are not 
particularly intelligent. They process vast amounts of data, seek patterns in them, and become adept at 
generating statistically probable results, such as human language and thought. However useful these 
programs may be in certain narrow areas, from linguistics and the philosophy of knowledge, we know 
how far they are from the way humans think and communicate (Chomsky, Roberts and Watumull, 2023). 

Unlike ANI, the human mind, which AGI aims to reach and ASI aims to far surpass, is not a clumsy 
statistical machine that absorbs hundreds of thousands of terabytes of data and costs (will cost) hundreds 
of billions of dollars just to arrive at the most probable answer to a trivial question. The human mind is a 
surprisingly efficient and elegant system for working with small amounts of information. It does not seek 
to build rough correlations between specific inputs but rather to provide explanations. Jeffrey Watumull 
argues that ANI programs are “stuck in the pre-human or non-human phase of cognitive evolution. Their 
deepest flaw is the lack of the central critical ability of any intellect: to say not only what is, what has been, 
and what will be but also what is not, what could be, and what cannot be. These are the ingredients of 
explanation, the hallmarks of true intelligence” (Chomsky, Roberts and Watumull, 2023). 

Although the development and general concept of AGI cannot directly and simply continue from the 
achieved level of ANI, ANI will nevertheless leave a significant corpus of its achievements as a legacy to 
higher forms of Al. Despite the ontological difference between the two Al systems, higher forms of Al will 
not abandon advanced deep learning algorithms, nor will they ignore the massive datasets already stored 
by ANI systems, even if they are stored only for a specific task. AGI will be intelligent enough to use that 
data for other purposes. The vast experience ANI programs will gain by answering millions of questions 
and solving millions of operational requests could be the virtual counterpart of the collective unconscious 
in humans, as the total record of the quantum of knowledge acquired by all people who have ever lived 
on the planet. AGI will be intelligent enough to unlock the treasures of that virtual collective unconscious 
acquired through the use of ANI. 

While in its responses to queries it cannot execute, ANI politely responds with learned phrases, 
uncomfortably, even foolishly ignoring the client for whom it exists, ANI is not aware of the limitations it has. 
Noam Chomsky and lan Roberts write how ANI foolishly “demonstrates the ‘banality of evil” (Smirnova, 
2023). It is assumed that AGI, self-constituting its strategy, will use the experiences of ANI to identify, 
understand, and overcome the limitations that ANI has failed to overcome. 

We are witnessing serious and competent debates “for and against’ Al. This is partly because Al is 
anew technology that people fear simply because it is new and changes their usual way of life. However, 
despite objections, ANI systems are becoming widely accepted and useful, which could lead people in the 
future to be more open to higher forms of Al. Higher forms of Al should be able to act on ethical principles, 
whether they be deontological or teleological moral principles. Experiences with ANI are completely 
useless in this area because all known forms of ANI are not able to understand or balance creativity and 
ethical constraints on their own, not even able to distinguish possible from impossible, which is not a good 
recommendation for higher forms of Al. The amorality, pseudoscience, and linguistic inadequacy of ANI 
make it either excessively produce both truths and lies, equally support ethical and unethical decisions or 
avoid making decisions and remain indifferent to the consequences of such attitudes. Given the amorality, 
pseudoscience, and linguistic simplicity of ANI systems, we can only mock or mourn their popularity 
(Chomsky, Roberts and Watumull, 2023). 


Discussion 


In the previous chapters, we explored the concept of “evil” in the context of Narrow Artificial 
Intelligence (ANI) or artificial intelligence (Al) in general. While ANI itself lacks consciousness or free will, 
its potential for misuse or harmful actions that usually result in damage and suffering to humans poses a 
significant ethical and legal challenge. In this chapter, we continue our analysis and discuss fundamental 
aspects of this controversial issue. Everything that happens on Earth is caused by either nature or people. 
Therefore, the crucial aspect of the potential “evil” in ANI comes from human decisions and intentions. 
ANI systems are inert and perform tasks according to their programming or training on examples and data 
presented to them during software development and subsequent testing. Any negative consequences of 
ANI can mostly be attributed to human decisions. This includes programming ANI algorithms, training 
models with biased or incomplete data, and decisions about the use of ANI in specific contexts. 

To understand and control the potential for “evil” use of ANI, it is important to analyze the role of 
programmers, engineers, and other Al industry professionals. Programmers have a significant influence on 
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how ANI behaves, even though they are often unaware of all the implications of their decisions. Therefore, 
it is essential to educate programmers and engineers about the ethical aspects of ANI and provide them 
with tools to identify and address potential issues. 

One of the common ethical challenges related to ANI is bias and discrimination in software designed 
to make decisions and perform tasks without human verification. ANI systems can inherit biases present 
in the data sets provided to them by administrators. This can result in unfair decisions, discrimination, 
and imbalances in the treatment of different groups of people. Research has shown that, despite the 
fact that Al’s abilities in processing data far exceed human capabilities in terms of speed and volume, 
ANI cannot always be trusted as fair and neutral. The underlying cause of ANI bias is linked to historical 
human prejudices (Lifshitz, 2021). Human biases are deeply ingrained and extend to certain groups of 
people, and these biases can be reinforced within computer models. Al systems, therefore, perpetuate 
existing biases in fields including healthcare, criminal justice, and education. Cases like those of the 
COMPAS algorithm (Correctional Offender Management Profiling for Alternative Sanctions) in the United 
States, which is more likely to arrest black individuals due to historical racism and differences in policing 
practices, highlight the need for more consistent regulation and ethical guidelines for the development and 
use of ANI, especially in sensitive sectors like security and justice (see more: Bjelajac and Filipovi¢, 2021). 

Another crucial aspect of the discussion about the potential “evil” use of ANI is the autonomous 
nature of some ANI systems. Autonomous military drones can be programmed to execute destructive tasks 
without human intervention. This raises profound ethical and moral questions in the military application of 
ANI. To limit the potential abuse of autonomous Al systems, clear ethical guidelines and regulations are 
needed in the military sector. These guidelines should define the boundaries of autonomous ANI operations 
and ensure that human responsibility and oversight are preserved. However, this will likely remain mostly 
declarative, as the military, by default, acts against enemies in warfare, and in such contexts, rules and 
laws have limited applicability. 

One of the key elements in preventing the potential “evil” use of ANI is transparency. Organizations 
developing ANI systems should be transparent about how they trained models and what data was used. 
This allows independent experts and organizations to audit and assess ANI systems to ensure they 
do not contain biases or malicious intentions. Additionally, it is important to establish mechanisms of 
accountability for Al systems. If irregularities or harm arise from the use of ANI, responsible parties, 
whether they are programmers, organizations, or owners of ANI systems, should face consequences. 
This involves the establishment of clear regulations and laws defining responsibility in case of issues with 
ANI systems. 

The balance between progress and risk is often cited as the most important dilemma of the future 
of Al. People fear non-human entities that could get out of control and begin to act as superhumans. 
This is where the story of freedom and free will, or moral good and moral evil, in the world of artificial 
intelligence begins. These two noumena are proportionally dependent. The more one grows, the other 
diminishes (see more: Kant, 1981). Therefore, governments of major countries want to maximize control 
over the development and implementation of artificial intelligence. The fundamental thesis with which the 
European Union approaches thinking about Al is: “We are building trust in artificial intelligence, and that is 
possible only if we are able to manage risks” (Riegert, 2021). ANI brings tremendous potential to address 
complex problems and improve human lives. In medicine, science, transportation, and many other areas, 
ANI has the ability to accelerate progress and make daily life easier for people. Hence, the challenge 
lies in finding a balance between the potential for positive impact of ANI and the need to limit its negative 
consequences. 


Conclusions 


Without diminishing the manifold advantages ushered in by contemporary technologies, the scientific 
community, with the exception of a substantial cohort encompassing roboticists, electronics specialists, 
and similar domains, is not reticent in expressing deep-seated apprehension concerning the burgeoning 
development of artificial intelligence (Al) and its escalating foray into pivotal sectors of public and private 
life. The deployment of systems founded on deep learning and self-learning neural networks, coupled with 
the utilization of machines that, on numerous occasions, outpace human counterparts in data analysis 
and expeditious decision-making, has led to concerns articulated by scientists, humanists, and futurists. 
Over time, these technologies are not merely poised to displace human labor in many employment sectors 
but also possess the potential to predict human consumer inclinations, modus operandi, communication 
patterns, and even exert influence over individual destinies, thereby encroaching upon the sacrosanct 
citadel of privacy. 
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Scientists posit that the concentration of metadata, power, and affluence in the hands of a select 
few has the capacity to obfuscate and render the entire sociotechnical system non-transparent and 
“opaque.” This eventuality is unequivocally predicted to give rise to heightened sociopolitical schisms 
and to exacerbate, potentially leading to a draconian breach of democratic rights and personal freedoms 
for both citizens and nations (Couldry and Meijas, 2019). Acknowledging that contemporary humanity, at 
this stage of its historical progression, evinces a predilection for the encapsulation of its essence within 
the realm of technology, scientists, particularly those within the domains of philosophy, sociology, and 
theology, evince reluctance in relegating the stewardship of Al ethics to engineers and technicians. The 
skepticism directed towards emerging technologies, particularly cognitive Al, can be expounded upon by 
recourse to the prevailing societal concept of scientific neutrality, while differentiating the jurisdictions of 
the scientific and legal spheres. It is our contention that the world is on the cusp of an academic schism, 
a tug-of-war between champions of raw power and proponents of global security and the preservation 
of “life as we know it.” This would not mark the inception of such a schism but would be resolved as in 
instances past—with the facile triumph of power’s apologists. The conventional narrative dictates that this 
innocuous dominion is destined to expeditiously metamorphose into financial and military might. This is 
not indicative of the world teetering on the precipice of dissolution; it serves as a clarion call for judiciously 
overseeing the use, and potential abuse, of Al. History has persistently underscored that behind every 
perilous contraption lurks an equally pernicious human agent. Our elucidation of this ethical aporia can be 
attributed to divergent interpretations regarding the core tenets of Al ethics. Practitioners from the realm 
of the natural sciences are preoccupied with the transliteration of prevailing, and occasionally antiquated, 
ethical paradigms into the vernacular of machinery. In contrast, philosophers, sociologists, humanists, and 
theologians grapple with the very essence of Al ethics, which must undergo an evolution commensurate 
with the assimilation into its ontological and metaphysical corpus of a conspicuous novelty—an artificial 
entity furnished with its heretofore alien concepts of good and evil. This nascent ethical framework shall be 
denominated a biomimetic ethos, one that may be adjusted on the fly by humans to conform to emerging 
entities, as this novel ethical framework is co-created in tandem with these entities. 

Artificial intelligence steadfastly advances, and each new application of this technology offers a 
novel aperture for autonomous systems to harness data to effect harmful outcomes. The promulgation 
of legislation emphasizing responsibility, transparency in Al, mitigation of bias, and the comprehensive 
implementation of ethical precepts is posited as an efficacious strategy for contending with the “dark 
Al.” The urgency of addressing the darker facets of Al mounts with each passing second. The present 
juncture, more than any other, underscores the need to glean instructive insights from humanity's historical 
missteps and prepare judiciously for the impending challenges. 
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Research and Training Center on Independent Living. (1993). Guidelines for reporting and writing 
about people with disabilities (4" ed.) [Brochure]. Lawrence, KS: Author. 

* The work of famous authors downloaded electronically applies the following form: 


The author’s name [comma], initial / names [point], [open small brackets] year of publication [close 
little brackets] [point] title of the paper [point], an indication of what kind of material is in square brackets, 
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taken (note that work will take) the day, month and year, with (internet address). 


Schwarzer, R. (1989). Statistics software for meta-analysis [Computer software and manual]. 
Retrieved March 23, 2001, http://www.yorku.ca/faculty/academic/schwarze/meta_e.htm 


* When the list reference is made to the work that is being prepared for the press, after the authors’ 
names, in parentheses, listed in the press in English. 


Zuckerman, M. Kieffer, SC (in press). Race differences in faceism: Does facial prominence imply 
dominance? Journal of Personality and Social Psychology. 

* When the list of references cites a newspaper article without the author prints the name of the 
article, then the time of publication, then the title and number - in italics, and at the end of the page on 
which the article was published. If the title is long, we can shorten the optimum number of words by taking 
the first few words. 


The new health-care lexicon. (1983, August / September). Copy Editor, 4, 1-2. 


¢ If within the journal as publisher publishes a special issue as a monograph, it is necessary after 
heading indicate that it is a monograph. 


Ganster, DC, Schaubroeck, J. Sime, WE, & Myers, BT (1991). The nomological validity of the Type 
A personality among employed adults [Monograph]. Journal of Applied Psychology, 76, 143-168. 


« When an abstract or summary of the quote as the original source, after the title should be in 
parentheses to indicate that it is abstract. 


Woolf, NJ, Young, SL, Famselow, MS, & Butcher, LL (1991). Map-2 expression in cholinoceptive 
pyramidal cells of rodent cortex and hippocampus is altered by Pavlovian conditioning [Abstract]. Society 
for Neuroscience Abstracts, 17, 480 harvesters. 


* Titles that are not in English, and we want them to be published in the journal in English, listed in 
their native language, and then in the square brackets give the title translation into English. In addition to 
the title, everything else remains the mother tongue. 


Ising, M. (2000). Intensitatsabhangigkeit evozierter Potenzial their EEG: Sindh impulsive persons 
Augmenter stage Reducer? [Intensuty dependence and event related EEG potentials: Are impulsive 
individuals augmenters or reducers?]. Zeitschrift fur Différentiel und diagnostisch Psychology, 21, 208-217. 


* In the list of literature translated work following a text that we have a year of the original edition 
listed in parentheses at the end behind the publisher. When we quote in plain text, year of first publication 
and translation writing along with a slash between (eg. Laplace, 1814/1951). 


Laplace, P. S. (1951). A philosophical essay on probabilities (FW Troscott & FL Emory, Trans.). New 
York: Dover. (Original work published 1814) 


* When the list of references cites a paper published in the Proceedings of the translated, italics will 
print the name of the collection at the end to add when it published the original. 


Freud, S. (1961). The ego and the id. In J. Strachey (Ed. & Trans.), The standard edition of the 
complete psychological works of Sigmund Freud (Vol. 19, pp. 3-66). London: Hogarth Press. (Original 
work published 1923). 


* When you cite articles published on the university or one of the official institutions, universities, 
publishers listed as the first name of the university and then university. 


Broadhurst, RG, & Maller, RA (1991). Sex offending and recidivism (Tech. Rep. No. 3). Nedlands: 
University of Western Australia, Crime Research Center. 
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* When the list of sources cites a report of an organization or institution that has no author, it is best 
to nominate as the author of this organization, which is also the publisher. 


Employee Benefit Research Institute. (1992, February). Sources of health insurance and 
characteristics of the uninsured (Issue Brief No. 123). Washington, DC: Author. 


* When the work was published on the Internet as a photocopy, it should cite the original source 
noting that this is the electronic version. 

Vandenbos, G. Knapp, S., & Doe, J. (2001). The role of reference elements in the selection of 
resources by psychology undergraduates [Electronic version]. Journal of Bibliographic Research, 5, 
117-123. 


* If you download from the Internet work that you believe is different from the original, do not copy 
or no numbered pages, then at the end indicate the date of downloads and web address. 


Vandenbos, G. Knapp, S., & Doe, J. (2001). The role of reference elements in the selection of 
resources by psychology undergraduates [Electronic version]. Journal of Bibliographic Research, 5, 117- 
123. Retrieved October 13, 2001, from http://jbr.org/articles.htm| 


* When you download from the Internet a document which has no date or author, then the document 
name takes the place of the author or the first place. 

8th GVU’s WWW User Survey. (Od). Retrieved August 8, 2000, from http://www.cc.gatech.edu/ 
gvu/user_surveys/survey-1997-10/ 


¢ Material from the symposium or a scientific paper which was only exposed, but not published, 
listed with the note on which the scientific or professional meeting is material exposed. If the author has 
presented on the site, it is desirable to name and web page. 


Cuter, LD, Frolich, B., & Hanrahan, P. (1997, January 16). Twohanded direct manipulation on the 
responsive workbench. Paper presented at the 1997 Symposium on Interactive 3D Graphics. Abstract 
retrieved June 12, 2000, from http:/Avww.graphics.standard.edu/papers/twohanded/ 


* Computer software listed noting computer software. Name of the software we write italics. 


Miller, M. E. (1993). The Interactive Tester (Version 4.0) [Computer software]. Weastminster, CA: 
Psytek Sesvice. 


* Data downloaded from the website of the government or other official organization listed noting 
data file. The filename of the data listed in italics. 


Department of Health and Human Services, National Center for Health Statistics. (1991). National 
Health Provider Inventory: Home health agencies and hospices, 1991. [Data file]. Available from the 
National Technical Information Service Web site, htto://www.ntis.gov 


Standards take according to Suzic, N. (2010). Pravila pisanija naucnog rada APA i drugi standardi 
[Rules scientific APA work and other standards]. XBS Banja Luka. 


Reference management software 

The journal has a standard template — |IJCRSEE template. To cite reference, it is the easiest way to 
use some management software like: 

Mendeley (http:/Avww.mendeley.com/features/reference-manager) 

EndNote (http://www.endnote.com/support/enstyles.asp) and 

Reference Manager (http://refman.com/support/rmstyles.asp). 

Above management software have plug-ins to word processing where authors only need to select 
the appropriate journal template when preparing their article and the list of references and citations to 
these will be formatted according to the journal style as described in this Guide. If you cannot find an 
available template, see the list of sample references and citations provided in this Guide to help you 
format these according to the journal style. 
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After acceptance 

Online proof correction 

Corresponding authors will receive an e-mail with a link to our online proofing system, where have 
to make prof own article. The final version is created in PDF and authors have to accept final version or 
to immediately report the error. We will do everything possible to get your article published quickly and 
accurately. Please check carefully before replying, as the inclusion of any subsequent corrections cannot 
be guaranteed. Proofreading is solely your responsibility. 

https://pubpeer.com/Check the comments on published and withdrawn works. 


All the articles in this journal have been peer reviewed. Nonetheless, editors and organizers are not 
responsible for the content shown in this publication. 


Submission Preparation Checklist 

As part of the submission process, authors are required to check off their submission’s compliance 
with all of the following items, and submissions may be returned to authors that do not adhere to these 
guidelines. 


The submission has not been previously published, nor is it before another journal for consideration 
(or an explanation has been provided in Comments to the Editor). 

The submission file is in OpenOffice, Microsoft Word, RTF, or WordPerfect document file format. 

Where available, URLs for the references have been provided. 

The text is single-spaced; uses a 12-point font; employs italics, rather than underlining (except 
with URL addresses); and all illustrations, figures, and tables are placed within the text at the appropriate 
points, rather than at the end. 

The text adheres to the stylistic and bibliographic requirements outlined in the Author Guidelines, 
which is found in About the Journal. 

If submitting to a peer-reviewed section of the journal, the instructions in Ensuring a Blind Review 
have been followed. 

Have checked paper for possible accidental plagiarism. Some plagiarism checker websites includes: 
http:/www.ithenticate.com, www.grammarly.com,www.plagtracker.com, or www.duplichecker.com 

To verify originality, your article may be checked by the originality detection service CrossCheck. 
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